# Portable deployment and recovery runbook (Phase 15)

## Configuration

Copy `.env.example` to an untracked deployment environment file. Production requires a secret-manager supplied `SESSION_SECRET` of at least 32 characters and refuses `AUTOMATED_OUTREACH_ENABLED=true`. Keep bootstrap credentials one-time only; remove and rotate them after provisioning. Never place secrets in images, Compose YAML, logs, backups, or public web roots.

Validate before startup:

```sh
python3 -c 'from app.config import load_config; load_config()'
docker compose config --quiet
```

## Health and readiness

- `GET /api/v1/health/live` is process liveness and unauthenticated.
- `GET /api/v1/health/ready` checks SQLite connectivity and returns HTTP 503 until ready.
- `scripts/healthcheck.sh` checks readiness and the outreach safety flag.

Use readiness for load balancers and container health checks; liveness is only for process supervision.

## SQLite backup and restore

Backups are host-side artifacts and never include `.env` or secret files. `backup_sqlite.sh` uses SQLite's online backup API for a consistent snapshot, writes with mode 0600 to a temporary file, atomically renames it, writes a SHA-256 sidecar, and retains only the newest configured count.

```sh
scripts/backup_sqlite.sh /var/lib/prospect-platform/prospects.db /var/backups/prospect-platform 30
```

Before restoring, stop application writes, verify the checksum sidecar, and use the explicit confirmation flag. The script first makes a pre-restore backup, then atomically replaces the target only after an integrity check:

```sh
scripts/restore_sqlite.sh /var/backups/prospect-platform/prospects-<timestamp>.db /var/lib/prospect-platform/prospects.db --confirm-restore
```

Validate backup directory permissions and keep copies encrypted/off-host according to the retention policy. Test restores in an isolated directory quarterly. Never use `docker compose down -v` on a data-bearing installation.

## Rollback

`scripts/rollback.sh` is deliberately non-destructive: it prints the approved image/tag or digest rollback procedure and executes no stop, delete, restore, or deployment action. Record old/new image digests, configuration revision, backup/checksum, and health/readiness evidence.

## systemd / Virtualmin

`systemd/prospect-api.service.example` is a least-privilege service example. Copy it to a reviewed systemd unit, create `/etc/prospect-platform/prospect.env` with mode 0600, use a dedicated user/data directory, and place TLS/reverse proxying in the Virtualmin-managed web tier. Do not put environment files under `public_html`.

## Monitoring and incident response

Monitor readiness failures, restart count, HTTP 5xx rate, SQLite backup age/checksum failures, disk usage, and unexpected outbound traffic. Alert when the latest backup is older than the agreed RPO or when a restore drill fails. Routine logs must not contain passwords, tokens, cookies, API keys, full contact values, or free-text notes.

On incident: record image/config revision and health state; preserve redacted logs and audit evidence; isolate the service for data loss, unauthorized access, or unexpected outbound traffic; rotate secrets through the secret manager; validate restore/readiness and tenant-scoped reads; then document root cause and retention impact. Outreach remains disabled throughout.