add ssrf-safe website analysis

docs/SECURITY.md

@@ -40,6 +40,17 @@ The MVP provides deterministic match suggestions, an explicit human confirmation
 
 No Phase 7 resolver, cache, or availability provider is enabled in the current Compose runtime. Before production, add egress/SSRF controls, provider and PSL update review, retention/deletion handling, monitoring, permission/audit coverage, and failure/rollback tests for all domain observations.
 
+## Phase 8 website-scanning controls
+
+- Scanning is an authenticated, tenant-scoped observation. Allow only `http` and `https`; reject credentials, unsupported schemes, malformed/localhost/single-label hosts, and disallowed IP literals. Never allow `file:`, `ftp:`, `gopher:`, `data:`, `javascript:`, or equivalent protocol smuggling.
+- Resolve immediately before connection and validate the actual destination address. Block loopback, private, link-local, multicast, reserved, and cloud-metadata ranges for IPv4 and IPv6. Re-run protocol, hostname, DNS, and IP checks on every redirect and protect against DNS rebinding; do not rely on an initial DNS check or an HTTP `Host` header.
+- Enforce hard budgets for connect/read/total time, response and decompressed bytes, retained body size, redirects, crawl depth/links, retries, and concurrency. Abort on budget exhaustion. Do not allow compression, redirects, or retries to bypass limits.
+- Fetch only explicitly allowed content types and links. Never submit forms, send user cookies/credentials/authorization headers, execute JavaScript, run plugins, or make arbitrary subresource requests. Treat fetched content and all TXT/HTML/script text as untrusted input and escape it on display.
+- Classify conservatively: `unknown`, `blocked`, `partial`, `timeout`, and `error` are not empty success and are not negative business facts. A classification is evidence of bounded content only—not ownership, identity, consent, deliverability, security, or permission to contact.
+- Persist scan history/cache with tenant isolation, normalized URL, policy/scanner version, redirect policy, observed time, freshness/expiry, applied budgets, and uncertainty/error metadata. Bound size/retention, redact secrets and response bodies, and invalidate/re-evaluate after policy, DNS, or scanner-version changes. A cache hit must be visibly non-fresh.
+
+No production-grade scanner egress proxy, isolated worker, or durable scan store is supplied by the current Compose runtime. Before enabling scanning in production, add SSRF/DNS-rebinding/redirect-chain tests, egress deny-by-default policy, abuse/rate controls, authenticated history authorization, retention/deletion, monitoring, and incident procedures. Scans must never cause form submission, acquisition, verification, enrichment, or outreach.
+
 ## Phase 5 source security controls
 
 Source adapters are a security boundary, not a generic fetch facility. Registry review must verify the source identity, terms/robots and licensing owner, permitted collection purpose, approval expiry, tenant scope, rate/concurrency budget, raw-record retention/deletion policy, and circuit thresholds. Keep these controls server-side and auditable; a UI flag or client-supplied source ID is not authorization.
@@ -59,7 +70,7 @@ If a future approved adapter fetches URLs, apply the SSRF requirements below in
 2. **MFA:** require phishing-resistant or TOTP MFA for administrator accounts in production, including the bootstrap admin before granting ongoing administrative access. Define recovery, enrollment, reset, and revocation procedures; do not treat a password-only bootstrap as production-ready.
 3. **Authentication and authorization:** enforce authorization server-side on every protected route, including every child-record, note, pipeline, and audit route. Rotate/regenerate sessions at login and privilege changes, expire idle/absolute sessions, revoke on logout/password reset, and test tenant isolation.
 4. **Cookies and CSRF:** use `HttpOnly`, `Secure` (production HTTPS), and an appropriate `SameSite` policy. Browser state-changing endpoints require CSRF tokens (or a rigorously reviewed equivalent); do not rely on CORS or cookie flags alone.
-5. **SSRF and future scanners:** no scanner is enabled in this release. If a future approved feature fetches a URL, allow only `http`/`https`, validate DNS/IP targets, block loopback/private/link-local/cloud-metadata ranges after resolution, limit redirects, enforce size/time limits, and re-check each redirect.
+5. **SSRF and scanner production hardening:** the Phase 8 scanner is bounded and conservative, but production still requires an egress proxy/isolation boundary, DNS-rebinding and redirect-chain regression tests, deny-by-default network policy, abuse controls, and durable scan-history/cache retention. Allow only `http`/`https`; never submit forms or execute JavaScript.
 6. **Input/output safety:** validate schema and content types, bound request and note/evidence sizes, parameterize database queries, escape output, and reject unsafe provenance URLs or markup. Treat operator-entered notes and sources as untrusted data.
 7. **Audit and retention:** the current audit/activity behavior is an MVP trail, not an immutable compliance log. Define append-only guarantees, retention, redaction/deletion rules, access controls, alerting, and export procedures before production.
 8. **Secrets:** inject production secrets from a secret manager or orchestrator secret store. Do not place them in images, Compose files, source, CI logs, or committed `.env` files. Remove bootstrap variables after first-run provisioning.