add durable jobs and live job monitor

README.md

@@ -1,6 +1,6 @@
 # Prospect Intelligence Platform
 
-A safety-first Phase 3 vertical slice for **manual**, evidence-led prospect qualification. It stores tenant-owned businesses and their child intelligence records, keeps provenance with each evidence item, supports a review pipeline, and records operationally relevant changes. **Automated discovery, DNS/website scanning, and outreach are not part of this release. Automated outreach is disabled.**
+A safety-first Phase 4 design/implementation boundary for **manual**, evidence-led prospect qualification and the future job/live-log workflow. The current runtime remains the Phase 3 manual vertical slice: it stores tenant-owned businesses and child intelligence records, keeps provenance with each evidence item, supports a review pipeline, and records operationally relevant changes. **Automated discovery, DNS/website scanning, and outreach are not part of this release. Automated outreach is disabled.**
 
 ## Included
 
@@ -11,8 +11,9 @@ A safety-first Phase 3 vertical slice for **manual**, evidence-led prospect qual
 - Responsive static dashboard under `apps/web` with authenticated explorer filters, paginated results, detail review, manual intake, notes/pipeline context, evidence provenance, and browser-only CSV preview.
 - Docker Compose runtime with non-root containers, read-only filesystems, health checks, and a named SQLite data volume.
 - Browser authentication with server-side sessions and an optional first-run admin bootstrap.
+- Phase 4 MVP job monitor and SQLite-backed job/event schema/API surface, with the production limitations documented below.
 
-## Phase 3 workflow
+## Current workflow and Phase 4 boundary
 
 1. A permitted workspace member manually creates or reviews a prospect.
 2. The business detail response is the aggregate record for that tenant; related intelligence/evidence rows are returned only through the tenant-scoped detail surface.
@@ -22,6 +23,14 @@ A safety-first Phase 3 vertical slice for **manual**, evidence-led prospect qual
 
 The API applies the organization/tenant boundary server-side to list, detail, child-record, notes, pipeline, and audit reads and writes. Clients must use the returned pagination metadata and follow `next`/`previous` links or tokens rather than assuming that one response contains the whole tenant dataset. See `apps/api/README.md` for the route contract and limits.
 
+### Phase 4 jobs/live logging contract
+
+The planned asynchronous contract is: create one tenant-scoped job, return a stable job identifier, and move it through `queued` → `running` → a terminal state (`succeeded`, `failed`, `cancelled`). Each accepted request should carry an idempotency key whose scope and request fingerprint prevent duplicate jobs while allowing a safe replay of the original result. A job should persist append-only events with a monotonically increasing per-job sequence number, timestamp, level/type, safe message, and job/tenant identifiers.
+
+Clients should poll a tenant-scoped job status/events endpoint using `after_sequence` (or an equivalent cursor), with bounded backoff and terminal-state handling. SSE is a planned low-latency delivery option, not a current implementation; polling remains the compatibility fallback. Cancellation and retry must be explicit, authorized controls: cancellation is cooperative and may finish as `cancelled` or report that the job is already terminal; retry creates a new attempt while retaining the original job/idempotency lineage and must not duplicate side effects.
+
+The current MVP has SQLite job/event persistence, job status/list/detail and event APIs, cancellation/retry controls, and a browser job monitor that polls while work is active. SSE is not implemented; it remains a future delivery optimization over the persisted cursor. There is no Redis/Celery worker: the current in-process worker is suitable only for development/pilot use and must not be treated as durable, horizontally scalable execution.
+
 ## Run locally
 
 ```bash
@@ -69,7 +78,7 @@ Authenticated browser requests use a server-side session cookie; login creates a
 
 ## Explicit non-goals and remaining limitations
 
-This Phase 3 release still has no automated discovery, DNS resolution, website/HTTP scanning, enrichment scheduler, external source adapter, email/SMS sender, or outreach endpoint. CSV remains a browser/API preview flow and does not silently persist rows. SQLite and the named local volume are suitable for the pilot only; there is no production migration runner, queue, or tested backup/restore command. The development password fallback is PBKDF2 rather than production Argon2id. Before production, complete the gates in `docs/SECURITY.md` and `docs/OPERATIONS.md`, including MFA, TLS, CSRF protection, rate limiting, durable audit retention, migrations, approved source policy, SSRF-safe fetching if a future scanner is approved, and tested backups/restores.
+This Phase 4 boundary still has no automated discovery, DNS resolution, website/HTTP scanning, enrichment scheduler, external source adapter, email/SMS sender, or outreach endpoint. CSV remains a browser/API preview flow and does not silently persist rows. SQLite, the in-process worker, and the named local volume are suitable for the pilot only; production migration, durable queue/worker leases, event retention/backup, SSE delivery, and tested backup/restore remain unfinished. Redis and Celery are not implemented. The development password fallback is PBKDF2 rather than production Argon2id. Before production, complete the gates in `docs/SECURITY.md` and `docs/OPERATIONS.md`, including MFA, TLS, CSRF protection, rate limiting, tenant-scoped job/event authorization, idempotent side-effect handling, durable audit/event retention, migrations, approved source policy, SSRF-safe fetching if a future scanner is approved, and tested backups/restores.
 
 ## Verification