add durable jobs and live job monitor

docs/OPERATIONS.md

@@ -31,6 +31,14 @@ Phase 3 is a human-operated prospect workflow. Operators manually create a busin
 
 There is no automated discovery job, DNS/website scanner, enrichment worker, or outreach worker to monitor in this release. CSV is preview-only; do not describe a preview as an import or assume that rows were persisted.
 
+## Phase 4 jobs and live logging
+
+The Phase 4 MVP provides SQLite-backed job status/detail/event routes and a browser monitor. A job moves `queued` → `running` → `succeeded`/`failed`/`cancelled`, retains its attempt and tenant identity, and appends per-job events with a monotonic sequence cursor. Operators inspect status and replay events by polling; SSE may provide lower-latency delivery but is not implemented and must replay from the persisted cursor and fall back to polling after disconnects.
+
+Creation must use a tenant-scoped idempotency key and request fingerprint. A repeated identical request returns the existing job/attempt; a conflicting payload is rejected. Cancellation is cooperative and race-safe, while retry is an explicit authorized new attempt linked to the original job and must not repeat completed side effects. Do not treat HTTP acceptance as completion, and do not reconstruct history from ephemeral container logs.
+
+The MVP has no SSE handler, durable queue, or worker process in Compose; its in-process worker and SQLite job/event tables are pilot-only. Process loss can lose work, there is no durable lease/recovery or horizontal coordination, and it must not be presented as production execution. Redis and Celery are not implemented.
+
 ## Configuration and deployment
 
 Copy `.env.example` for local development. Production values must be supplied by the deployment environment, never committed. `BOOTSTRAP_ADMIN_EMAIL` and `BOOTSTRAP_ADMIN_PASSWORD` are optional API environment variables for first-run admin provisioning only; set them together through a secret store or protected deployment environment, remove them immediately after successful bootstrap, and rotate the password. Do not put real values in Compose files, CI variables visible to logs, images, or committed `.env` files.
@@ -56,6 +64,8 @@ Inspect the volume with `docker volume inspect prospect-platform-api-data`; do n
 
 For the current MVP there is no database migration or backup command. If runtime data is material, stop writes first and snapshot/copy the volume using an approved host backup process. Protect business, child intelligence, notes, provenance, and audit data with encryption and access controls, test a restore into an isolated environment, and document the result. Define retention/deletion rules that cover source references and notes as well as contact fields.
 
+When jobs are introduced, backups and retention must include job definitions, idempotency records, terminal results/errors, and persisted sequence events. Verify that restoring a database preserves event ordering/cursors and does not cause a retried worker to repeat side effects. Define event redaction and retention separately from short-lived delivery connections.
+
 Recommended starting policy for a future production data store:
 
 - daily encrypted backups, with at least 30 days of retention;
@@ -75,9 +85,9 @@ Do not run `docker compose down -v` on a data-bearing environment: it removes th
 - **Build failure:** run `docker compose build --no-cache` from a reviewed checkout and check Docker daemon/network status.
 - **Unexpected outbound traffic:** stop the stack, preserve logs/metadata, and investigate. The MVP has no outreach worker and must not send automated messages.
 
-## Scaling path
+## Production migration and scaling path
 
-Adding Postgres, Redis, workers, schedulers, discovery adapters, or scanners requires explicit readiness checks, migrations, queue durability/idempotency, secret injection, network segmentation, metrics/alerts, backup/restore procedures, provenance/source governance, and an operational owner. Do not add them as an implicit Compose dependency: this MVP is intentionally runnable without external Postgres or Redis, and no automated discovery or outreach may be inferred from the scaling path.
+Before production, complete a migration from SQLite to a reviewed production database, add schema/indexes for jobs/idempotency/events, implement transactional sequence assignment and tenant authorization, and prove cancellation/retry/lease recovery under concurrency. Add durable queue/worker operations, metrics and alerts for queue age, failures, retries, cancellation latency, event lag/gaps, and SSE connections; define backup/restore and event-retention drills. Redis, Celery, Postgres, schedulers, discovery adapters, and scanners are possible future components—not implicit Compose dependencies and not implemented by this MVP. No automated discovery or outreach may be inferred from the scaling path.
 
 ## Incident checklist