Runbooks

Operational runbooks live next to the backend code, in NexisOmni/docs/ops/. This page indexes them and captures the cross-cutting operational gotchas worth knowing before you touch a live environment.

Runbooks in the backend repo

• Tenant graduation (docs/ops/tenant-graduation.md) - promoting a shared tenant to a dedicated database, including the crash-recovery procedure.
• Deploy runbook (docs/ops/deploy-runbook.md) - the hand-rolled Caddy + docker-compose stack (an alternative to the Dokploy path in Deploying).

Recurring gotchas

Data-protection key ring must be persisted

In production the API only warns (it does not fail) when the key-ring path is unset or wrongly mounted. A bad mount silently uses an ephemeral ring, and every encrypted tenant connection string becomes unreadable on the next restart. Confirm the key file is written to its volume before provisioning any tenant.

Transactional email has DNS lead time

Password reset depends on transactional email (Resend). SPF and DKIM records take time to propagate, so treat email domain verification as a hard prerequisite, set up ahead of a go-live, not a last-minute step.

Back up both the database and the key ring

A database backup alone is not a restore. The encrypted connection strings are only decryptable with the data-protection key ring, so back up the key ring together with the database dump.

Windows dev-host gotcha

If Docker Desktop restarts, the host port 5432 can land inside a Windows excluded port range, and PostgreSQL then cannot publish it. The fix is to reserve the port explicitly with a persistent netsh int ipv4 add excludedportrange entry after stopping WinNAT.

Where new runbooks go

Verified, step-by-step operational procedures (provisioning a tenant, rotating the JWT key, restoring from backup) will be collected here as they are written and tested.