add ssrf-safe website analysis

docs/OPERATIONS.md

@@ -61,13 +61,21 @@ Review domain-to-business candidates separately from DNS results. Verify tenant
 
 Availability is `unknown` unless the API reports a result from an authorized provider. Before enabling one, verify current product/legal/security approval, terms owner, allowed tenant scope, request/concurrency and timeout limits, retention/deletion class, health/circuit state, and explicit operational enablement. `nxdomain`, `no_data`, timeout, stale cache, or provider error is not “available.” On provider outage, rate-limit, terms/approval expiry, circuit-open, or conflicting result, fail closed and report unknown/deferred; never purchase, reserve, contact, or retry through an unapproved provider.
 
+## Phase 8 website-scanning operations
+
+Website scans are bounded evidence collection, not browser sessions. The API must accept only `http`/`https`, resolve and validate DNS/IP immediately before connection, block loopback/private/link-local/multicast/reserved/cloud-metadata destinations, and repeat those checks for every redirect. Review redirect chains for protocol or host-policy violations; DNS rebinding or an unsafe resolved address is a blocked scan, not a retry opportunity.
+
+Monitor per-tenant and global scan counts, active concurrency, queue age, total/connect/read timeouts, response/decompressed bytes, redirect and crawl depth/link counts, cache hit/freshness, blocked destinations, DNS failures, partial results, and classification/error rates. Enforce hard budgets across redirects and discovered links. Classifications are conservative observations only: never submit forms, send cookies/credentials, execute JavaScript, or treat an HTTP 200/empty page as proof of a business fact. A scan that hits a limit is explicitly incomplete/unknown.
+
+Scan history and cache entries must retain normalized URL, redirect chain, observed time, scanner/policy version, applied budgets, freshness/expiry, cache status, and uncertainty reasons, with tenant authorization on every read. Keep retention and size bounded; redact response bodies, secrets, cookies, authorization data, and unnecessary personal data. Invalidate or re-evaluate entries when scanner/DNS policy changes. On SSRF indicators, unexpected egress, repeated budget abuse, or unsafe redirect chains, stop/disable scanning, preserve safe metadata, and follow the incident checklist.
+
 ## Phase 4 jobs and live logging
 
 The Phase 4 MVP provides SQLite-backed job status/detail/event routes and a browser monitor. A job moves `queued` → `running` → `succeeded`/`failed`/`cancelled`, retains its attempt and tenant identity, and appends per-job events with a monotonic sequence cursor. Operators inspect status and replay events by polling; SSE may provide lower-latency delivery but is not implemented and must replay from the persisted cursor and fall back to polling after disconnects.
 
 Creation must use a tenant-scoped idempotency key and request fingerprint. A repeated identical request returns the existing job/attempt; a conflicting payload is rejected. Cancellation is cooperative and race-safe, while retry is an explicit authorized new attempt linked to the original job and must not repeat completed side effects. Do not treat HTTP acceptance as completion, and do not reconstruct history from ephemeral container logs.
 
-The MVP has no SSE handler, durable queue, or worker process in Compose; its in-process worker and SQLite job/event tables are pilot-only. Process loss can lose work, there is no durable lease/recovery or horizontal coordination, and it must not be presented as production execution. Redis and Celery are not implemented.
+There is no SSE handler, durable queue, scan worker/isolation boundary, or worker process in Compose; its in-process worker and SQLite job/event tables are pilot-only. Process loss can lose work, there is no durable lease/recovery or horizontal coordination, and it must not be presented as production execution. Redis and Celery are not implemented.
 
 ## Configuration and deployment
 
@@ -117,7 +125,7 @@ Do not run `docker compose down -v` on a data-bearing environment: it removes th
 
 ## Production migration and scaling path
 
-Before production, complete a migration from SQLite to a reviewed production database, add schema/indexes for jobs/idempotency/events and domain observations, implement transactional sequence assignment and tenant authorization, and prove cancellation/retry/lease recovery under concurrency. Add durable queue/worker operations, bounded DNS/PSL processing, TTL-aware cache invalidation, uncertainty and association-review workflows, and a separately approved availability provider. Add metrics and alerts for queue age, failures, retries, cancellation latency, event lag/gaps, DNS status/error rates, cache freshness, provider rate limits/circuit state, and SSE connections; define backup/restore and event-retention drills. Redis, Celery, Postgres, schedulers, discovery adapters, and scanners are possible future components—not implicit Compose dependencies and not implemented by this MVP. No automated discovery, domain acquisition, ownership assertion, or outreach may be inferred from the scaling path.
+Before production, complete a migration from SQLite to a reviewed production database, add schema/indexes for jobs/idempotency/events, domain observations, and scan history/cache, implement transactional sequence assignment and tenant authorization, and prove cancellation/retry/lease recovery under concurrency. Add durable queue/worker and scanner-isolation operations, bounded DNS/PSL/website processing, TTL/freshness-aware cache invalidation, SSRF/DNS-rebinding/redirect-chain tests, hard size/time/crawl budgets, uncertainty and association-review workflows, and a separately approved availability provider. Add metrics and alerts for queue age, failures, retries, cancellation latency, event lag/gaps, DNS/scanner status/error rates, cache freshness, blocked destinations, crawl-budget exhaustion, provider rate limits/circuit state, and SSE connections; define backup/restore and event-retention drills. Redis, Celery, Postgres, schedulers, discovery adapters, and production scanners are possible future components—not implicit Compose dependencies. No automated discovery, domain acquisition, ownership assertion, or outreach may be inferred from the scaling path.
 
 ## Incident checklist