expand manual prospect intelligence workflows

docs/OPERATIONS.md

@@ -18,19 +18,33 @@ The expected health endpoints are:
 
 A service is ready only when Compose reports `healthy`; container running status alone is insufficient. Health checks call public liveness endpoints and must remain unauthenticated—do not add a session requirement to `/api/v1/health/live` or `/healthz`. The Compose environment explicitly carries `AUTOMATED_OUTREACH_ENABLED=false` as an operational safety setting.
 
+## Phase 3 workflow operations
+
+Phase 3 is a human-operated prospect workflow. Operators manually create a business, add child intelligence/evidence observations with their provenance, review the detail page, add notes, and move the prospect through the permitted pipeline states. The API records the acting user and tenant on state-changing actions and exposes bounded activity/audit history where configured.
+
+- Treat source/provenance fields as required lineage for manual evidence: retain the source reference or label, captured/observed time, and confidence/context supplied by the operator.
+- Review pagination metadata and filters when investigating a list. Never infer that a page is the complete tenant dataset, and never use a UI filter as proof of authorization.
+- Investigate a missing detail or child record as a possible tenant/parent scope issue before retrying with alternate IDs. Cross-tenant IDs are expected to return not found.
+- Pipeline state is coordination metadata only. Suppressed records remain blocked, and no state enables outreach.
+- Notes may contain sensitive information. Limit access and avoid copying secrets, credentials, or unnecessary personal data into notes or audit details.
+- Audit/activity records are operational evidence of changes, not a replacement for a production-grade immutable audit service.
+
+There is no automated discovery job, DNS/website scanner, enrichment worker, or outreach worker to monitor in this release. CSV is preview-only; do not describe a preview as an import or assume that rows were persisted.
+
 ## Configuration and deployment
 
 Copy `.env.example` for local development. Production values must be supplied by the deployment environment, never committed. `BOOTSTRAP_ADMIN_EMAIL` and `BOOTSTRAP_ADMIN_PASSWORD` are optional API environment variables for first-run admin provisioning only; set them together through a secret store or protected deployment environment, remove them immediately after successful bootstrap, and rotate the password. Do not put real values in Compose files, CI variables visible to logs, images, or committed `.env` files.
 
-For production, use Argon2id for password hashing and require MFA for administrator accounts. Configure TLS before enabling `Secure` session cookies. Local Compose uses HTTP, so browser testing of the production `Secure` cookie behavior requires an HTTPS staging environment. Treat session cookies as bearer credentials: protect state-changing routes with CSRF controls, expire/revoke sessions, and never print cookie values in logs.
+For production, use Argon2id for password hashing and require MFA for administrator accounts. Configure TLS before enabling `Secure` session cookies. Local Compose uses HTTP, so browser testing of production `Secure` cookie behavior requires an HTTPS staging environment. Treat session cookies as bearer credentials: protect state-changing routes with CSRF controls, expire/revoke sessions, and never print cookie values in logs.
 
 Before deployment:
 
-1. Run `docker compose config` and review the rendered configuration (optional bootstrap values should be empty in CI and local validation).
+1. Run `docker compose config` and review the rendered configuration; optional bootstrap values should be empty in CI and local validation.
 2. Build from a reviewed commit and scan the resulting images.
 3. Restrict host/network exposure at the ingress/firewall.
-4. Verify both unauthenticated health checks and review logs for unexpected errors or sensitive data.
-5. Record the image digest and configuration revision for rollback.
+4. Verify both unauthenticated health checks and review logs for unexpected errors, cross-tenant errors, or sensitive data.
+5. Exercise tenant-scoped list/detail/child routes with bounded pagination and filters, and verify that notes/pipeline changes appear in the intended tenant's audit trail only.
+6. Record the image digest and configuration revision for rollback.
 
 ## Data, backups, and retention
 
@@ -40,7 +54,7 @@ Any older files under `infrastructure/docker/` are not referenced by Compose and
 
 Inspect the volume with `docker volume inspect prospect-platform-api-data`; do not treat a local Docker volume as a backup.
 
-For the current MVP there is no database migration or backup command. If runtime data is material, stop writes first and snapshot/copy the volume using an approved host backup process. Protect backup files with encryption and access controls, test a restore into an isolated environment, and document the result.
+For the current MVP there is no database migration or backup command. If runtime data is material, stop writes first and snapshot/copy the volume using an approved host backup process. Protect business, child intelligence, notes, provenance, and audit data with encryption and access controls, test a restore into an isolated environment, and document the result. Define retention/deletion rules that cover source references and notes as well as contact fields.
 
 Recommended starting policy for a future production data store:
 
@@ -56,18 +70,20 @@ Do not run `docker compose down -v` on a data-bearing environment: it removes th
 
 - **Unhealthy API:** inspect `docker compose logs api`, verify port binding and resource availability, then restart with `docker compose restart api` if appropriate.
 - **Unhealthy web:** inspect `docker compose logs web`; confirm port `8080` is available and the image contains `/healthz`.
+- **Missing or inconsistent detail:** preserve the request identifiers/log context, verify the authenticated tenant and parent-child association, and do not retry by guessing another tenant's ID.
+- **Audit gap:** stop the affected mutation workflow, preserve the database/log evidence, and investigate before allowing operators to rely on the history.
 - **Build failure:** run `docker compose build --no-cache` from a reviewed checkout and check Docker daemon/network status.
 - **Unexpected outbound traffic:** stop the stack, preserve logs/metadata, and investigate. The MVP has no outreach worker and must not send automated messages.
 
 ## Scaling path
 
-Adding Postgres, Redis, workers, or schedulers requires explicit readiness checks, migrations, queue durability/idempotency, secret injection, network segmentation, metrics/alerts, backup/restore procedures, and an operational owner. Do not add them as an implicit Compose dependency: this MVP is intentionally runnable without external Postgres or Redis.
+Adding Postgres, Redis, workers, schedulers, discovery adapters, or scanners requires explicit readiness checks, migrations, queue durability/idempotency, secret injection, network segmentation, metrics/alerts, backup/restore procedures, provenance/source governance, and an operational owner. Do not add them as an implicit Compose dependency: this MVP is intentionally runnable without external Postgres or Redis, and no automated discovery or outreach may be inferred from the scaling path.
 
 ## Incident checklist
 
 1. Record time, affected service, image/config revision, and observed health state.
-2. Preserve relevant logs without exporting secrets or unnecessary contact data.
-3. Stop or isolate the affected service if data loss, unauthorized access, SSRF, or unexpected outreach is suspected.
+2. Preserve relevant logs and audit records without exporting secrets or unnecessary contact data.
+3. Stop or isolate the affected service if data loss, unauthorized access, SSRF, provenance tampering, or unexpected outreach is suspected.
 4. Rotate exposed credentials through the secret manager.
-5. Validate recovery with health checks and a targeted smoke test.
-6. Document root cause, corrective action, and any retention/suppression impact.
+5. Validate recovery with health checks and a targeted tenant-isolation/detail smoke test.
+6. Document root cause, corrective action, and any retention/suppression or audit impact.

docs/SECURITY.md

@@ -2,30 +2,35 @@
 
 ## Current safety boundary
 
-- **Automated outreach is disabled.** The compose file sets `AUTOMATED_OUTREACH_ENABLED=false` for both services. The MVP sends no email, SMS, or other outbound communication.
+- **Automated outreach is disabled.** The Compose file sets `AUTOMATED_OUTREACH_ENABLED=false` for both services. The MVP sends no email, SMS, or other outbound communication.
+- Phase 3 intelligence is **manual and provenance-first**. Operators enter child intelligence/evidence records; the platform does not perform automated prospect discovery, DNS resolution, website/HTTP scanning, or external enrichment.
+- Every business, child record, note, pipeline transition, and audit/activity read or write must be constrained to the authenticated user's organization. A child identifier must never bypass the parent/tenant check. Cross-tenant misses should be indistinguishable from an absent record.
+- Evidence provenance (source reference/label, captured or observed time, actor, and confidence where supported) is data lineage, not proof that the platform independently verified the source. Do not fabricate provenance or silently upgrade an observation to a verified fact.
+- List and child-record APIs use bounded pagination and server-side filters. Bounds must be enforced before query execution and filters must be combined with the tenant predicate; never use client-side filtering as an authorization control.
+- Pipeline and notes are collaboration metadata. A reviewed pipeline state does not authorize contact, and suppression takes precedence over every other state. Audit entries should capture actor, tenant, target, action, timestamp, and safe details without secrets or unnecessary contact data.
 - No credentials are committed. `.env.example` contains non-secret names and local defaults only.
 - Authentication uses server-side sessions for browser clients. The session identifier is carried in an `HttpOnly` cookie; logout/revocation must invalidate the server-side session. Health endpoints are deliberately public and must remain usable without a session.
 - Containers run as an unprivileged user, drop Linux capabilities, use `no-new-privileges`, and use read-only root filesystems. The API data volume is the only intended writable persistent location.
-- The stdlib API remains a small MVP security boundary. Authentication/session handling does not by itself provide authorization, CSRF protection, rate limiting, MFA, or a complete audit log.
 
 ## Known limitations before production
 
 1. **Password storage:** production passwords must be hashed with Argon2id using a reviewed cost/memory/parallelism policy. Never store plaintext or reversible passwords, and never log bootstrap credentials. Rehash on login when the policy changes.
 2. **MFA:** require phishing-resistant or TOTP MFA for administrator accounts in production, including the bootstrap admin before granting ongoing administrative access. Define recovery, enrollment, reset, and revocation procedures; do not treat a password-only bootstrap as production-ready.
-3. **Authentication and authorization:** enforce authorization server-side on every protected route, rotate/regenerate sessions at login and privilege changes, expire idle/absolute sessions, revoke on logout/password reset, and test tenant isolation. The bootstrap variables are one-time provisioning inputs, not a standing authentication mechanism.
-4. **Cookies and CSRF:** use `HttpOnly`, `Secure` (production HTTPS), and an appropriate `SameSite` policy. `Secure` cookies cannot be exercised over the local HTTP Compose URLs, and `SameSite` is defense-in-depth—not a complete CSRF control. Browser state-changing endpoints require CSRF tokens (or a rigorously reviewed equivalent); do not rely on CORS or cookie flags alone.
-5. **SSRF:** any future URL fetcher must allow only `http`/`https`, validate DNS/IP targets, block loopback/private/link-local/cloud-metadata ranges after resolution, limit redirects, enforce size/time limits, and re-check each redirect. Never fetch arbitrary user-provided URLs from the server without these controls.
-6. **Input/output safety:** validate schema and content types, bound request sizes, parameterize database queries, escape output, and avoid logging contact data or secrets.
-7. **Secrets:** inject production secrets from a secret manager or orchestrator secret store. Do not place them in images, Compose files, source, CI logs, or committed `.env` files. Remove bootstrap variables after first-run provisioning.
-8. **Transport and perimeter:** terminate TLS at a trusted ingress, restrict exposed ports, add network policy, and place admin surfaces behind appropriate access controls.
-9. **Data protection:** define retention and deletion rules for prospect/contact data, restrict volume access, encrypt backups, and maintain an access/audit trail.
+3. **Authentication and authorization:** enforce authorization server-side on every protected route, including every child-record, note, pipeline, and audit route. Rotate/regenerate sessions at login and privilege changes, expire idle/absolute sessions, revoke on logout/password reset, and test tenant isolation.
+4. **Cookies and CSRF:** use `HttpOnly`, `Secure` (production HTTPS), and an appropriate `SameSite` policy. Browser state-changing endpoints require CSRF tokens (or a rigorously reviewed equivalent); do not rely on CORS or cookie flags alone.
+5. **SSRF and future scanners:** no scanner is enabled in this release. If a future approved feature fetches a URL, allow only `http`/`https`, validate DNS/IP targets, block loopback/private/link-local/cloud-metadata ranges after resolution, limit redirects, enforce size/time limits, and re-check each redirect.
+6. **Input/output safety:** validate schema and content types, bound request and note/evidence sizes, parameterize database queries, escape output, and reject unsafe provenance URLs or markup. Treat operator-entered notes and sources as untrusted data.
+7. **Audit and retention:** the current audit/activity behavior is an MVP trail, not an immutable compliance log. Define append-only guarantees, retention, redaction/deletion rules, access controls, alerting, and export procedures before production.
+8. **Secrets:** inject production secrets from a secret manager or orchestrator secret store. Do not place them in images, Compose files, source, CI logs, or committed `.env` files. Remove bootstrap variables after first-run provisioning.
+9. **Transport and perimeter:** terminate TLS at a trusted ingress, restrict exposed ports, add network policy, and place admin surfaces behind appropriate access controls.
+10. **Data protection:** define retention and deletion rules for prospect/contact data and provenance, restrict volume access, encrypt backups, and maintain a tested access/audit trail.
 
 ## Source and contact policy
 
-Treat discovered business information as potentially personal or copyrighted data. Collect only what is needed for the documented product purpose, preserve source attribution where required, respect site terms and robots/access policies, and provide suppression/deletion handling. Do not infer consent to contact from public availability. Any future outreach feature requires an explicit product/legal review and must remain off by default.
+Treat manually supplied business information, notes, and source references as potentially personal or copyrighted data. Collect only what is needed for the documented product purpose, preserve source attribution where required, respect site terms and robots/access policies, and provide suppression/deletion handling. Do not infer consent to contact from public availability or a reviewed pipeline state. Any future outreach or discovery feature requires explicit product/legal and security review and must remain off by default.
 
 ## CI/dependency hygiene
 
-Pin or review base-image and dependency updates, scan images before release, use least-privilege GitHub tokens, and avoid printing environment values. CI may validate Compose with empty optional bootstrap variables and call unauthenticated health checks; it is not a substitute for Argon2id parameter review, MFA testing, or a security assessment.
+Pin or review base-image and dependency updates, scan images before release, use least-privilege GitHub tokens, and avoid printing environment values. CI may validate Compose with empty optional bootstrap variables and call unauthenticated health checks; it is not a substitute for authorization/tenant-isolation tests, provenance policy review, Argon2id parameter review, MFA testing, or a security assessment.
 
 Report vulnerabilities privately to the repository maintainers; do not include live credentials or personal data in an issue.