add production readiness and recovery assets

docs/OPERATIONS.md

@@ -204,3 +204,50 @@ On suspected suppression bypass, invalid consent/legal basis, cross-tenant expos
 4. Rotate exposed credentials through the secret manager.
 5. Validate recovery with health checks and a targeted tenant-isolation/detail smoke test.
 6. Document root cause, corrective action, and any retention/suppression or audit impact.
+
+## Phase 15 portable operations
+
+See `docs/DEPLOYMENT.md` for the production-safe environment contract, `/api/v1/health/ready` readiness semantics, atomic SQLite backup/restore procedures, checksum/retention policy, non-destructive rollback guidance, monitoring references, and the systemd/Virtualmin-compatible service example. Run `scripts/healthcheck.sh` for an operator-safe readiness probe. Backups are host-side and must remain encrypted/off-host; never include `.env` or secret-manager material.
+
+## Phase 15 production deployment runbook
+
+### Prerequisites and Virtualmin layout
+
+Use a dedicated, patched Linux VPS with Docker Engine and Compose v2, adequate disk/RAM/CPU, host firewalling, DNS control, HTTPS certificates with renewal monitoring, and an encrypted off-host backup destination. Virtualmin may host the domain and terminate TLS/reverse-proxy to Compose, but it does not replace Docker health checks, application authorization, backups, or monitoring. Keep the checkout and `.env` outside public web roots with restrictive permissions; expose only the reverse proxy publicly and keep the API binding private where the topology permits.
+
+### Configuration and first bootstrap
+
+```sh
+cp .env.example .env
+chmod 600 .env
+docker compose config --quiet
+docker compose up --build -d
+docker compose ps
+curl -fsS https://example.invalid/healthz
+curl -fsS http://127.0.0.1:8000/api/v1/health/live
+```
+
+Replace the example hostname with the real HTTPS origin. Supply secrets through the protected deployment environment/secret manager, not shell history or committed files. Set `BOOTSTRAP_ADMIN_EMAIL` and `BOOTSTRAP_ADMIN_PASSWORD` only for a fresh instance, confirm authenticated login, then remove both values and rotate the password. Verify the rendered config still contains `AUTOMATED_OUTREACH_ENABLED=false`; never override it as a routine deployment action.
+
+### Readiness, monitoring, and release evidence
+
+The API `/api/v1/health/live` is a liveness check and `/api/v1/health/ready` checks SQLite readiness; web `/healthz` is a liveness check. These endpoints do not prove backups, workers, migrations, or external dependencies, so readiness remains an operator gate: both services must report Compose `healthy`, HTTPS must reach the expected containers, authenticated tenant-scoped smoke tests must pass, and migration validation must be recorded. Monitor container health/restarts, CPU/RAM/disk and `/data` pressure, API latency/error rates, auth failures, backup age/failures, TLS expiry, and unexpected egress. Never log secrets, cookies, full contact values, or request bodies. Record the commit, image digests, rendered non-secret configuration fingerprint, schema/migration result, backup ID, and approver.
+
+### Backup, restore, retention, and migration validation
+
+The named volume `prospect-platform-api-data` is live state, not a backup. Before a release or schema change, quiesce writes, take an encrypted backup to an off-host/isolated destination, verify its checksum/manifest, and restore it into a disposable isolated volume. Run `PRAGMA integrity_check`, foreign-key checks, representative tenant-scoped API reads, row-count checks, and the API test suite against the restored copy. Record the result and retain the previous image/config. Apply documented retention to SQLite data, audit/source lineage, operational logs, and backup generations; honor legal holds and verify deletion jobs where present. Do not use `docker compose down -v` on a data-bearing environment.
+
+There is currently no standalone migration or backup CLI. `schema.sql` is applied by the API startup and additive compatibility behavior is in application code; therefore every schema change requires a reviewed backup-first procedure and isolated restore test. Do not assume startup success means migration success. Stop and roll back the release if integrity, tenant isolation, health, or smoke validation fails.
+
+### Rollback
+
+1. Stop promotion and record symptoms, health, commit/image/config revisions, and backup ID.
+2. Disable the affected Virtualmin route or put the site in maintenance mode; stop writes if data integrity is in doubt.
+3. Re-deploy the previously verified image pair and exact configuration. Do not run an older binary against a schema it cannot read.
+4. Re-run health, authenticated tenant-isolation smoke tests, and read-only integrity checks.
+5. Restore the database only when the backup/schema compatibility is verified and an incident owner approves it; otherwise preserve the newer data and perform forward repair.
+6. Re-enable traffic only after monitoring is green, then document root cause, retention/legal impact, and follow-up migration work.
+
+### Explicit limitations
+
+This repository does not provision Virtualmin/TLS/DNS, provide a dependency-aware readiness service beyond the API's SQLite check, durable migration runner, PITR, HA database, durable queue/worker leases, production egress proxy, or compliance-grade retention service. SQLite and the in-process worker are pilot-only. The release has no outbound provider/send path and must remain outbound-disabled by default. See `docs/RELEASE_CHECKLIST.md` for the short go/no-go gate.