add ssrf-safe website analysis

README.md

@@ -1,6 +1,6 @@
 # Prospect Intelligence Platform
 
-A safety-first Phase 7 design/implementation boundary for **manual**, evidence-led prospect qualification, controlled source ingestion, and domain intelligence review. The current runtime remains a manual vertical slice: it stores tenant-owned businesses and child intelligence records, keeps provenance with each evidence item, supports a review pipeline, and records operationally relevant changes. Phase 7 defines conservative registrable-domain/PSL and DNS observation semantics in addition to the Phase 6 normalization and deduplication rules; it does **not** enable network discovery. **Automated outreach is disabled, and no live source may be enabled without explicit approval.**
+A safety-first Phase 8 design/implementation boundary for **manual**, evidence-led prospect qualification, controlled source ingestion, bounded website scanning, and domain intelligence review. Phase 8 website scanning is a conservative observation workflow: it never submits forms, executes JavaScript, follows unsafe protocols, or authorizes outreach. **Automated outreach is disabled, and no live source may be enabled without explicit approval.**
 
 ## Included
 
@@ -80,7 +80,7 @@ Authenticated browser requests use a server-side session cookie; login creates a
 
 ## Phase 5 source boundary and remaining limitations
 
-Phase 5 defines a source adapter contract and registry; it does not implement network discovery, DNS resolution, website/HTTP scanning, enrichment scheduling, or a live external-source adapter. A source adapter must declare its identity, terms owner, permitted purpose, rate limits, retention class, query/result schema, dry-run behavior, and health/circuit controls. CSV and manual reference adapters may be used for operator-supplied data; they must preserve source attribution and raw source records, and must not silently turn preview data into outreach or verified facts.
+Phase 5 defines a source adapter contract and registry; Phase 8 adds a bounded website-observation adapter, but it does not implement general network discovery, enrichment scheduling, or a live external-source adapter. A source adapter must declare its identity, terms owner, permitted purpose, rate limits, retention class, query/result schema, dry-run behavior, and health/circuit controls. CSV and manual reference adapters may be used for operator-supplied data; they must preserve source attribution and raw source records, and must not silently turn preview data into outreach or verified facts.
 
 A discovery query is a tenant-scoped, bounded, auditable request that can be validated and dry-run without contacting a source. Any live source requires explicit product/legal/security approval, a registered adapter, and an operational enablement decision; absent all three, execution must fail closed. Circuit-open, rate-limit, terms, or approval failures must produce a safe non-live result. Raw source records are retained only under the approved retention class and must exclude secrets and unnecessary personal data.
 
@@ -108,6 +108,18 @@ The platform must not claim that a domain is available, unregistered, or safe to
 
 Phase 7 remains a documentation/contract boundary in this MVP: there is no live DNS resolver, PSL-backed enrichment worker, cache service, or availability provider in Compose. Production work still includes selecting and versioning the PSL, implementing bounded DNS resolution and TTL-aware cache invalidation, defining MX/NS/TXT parsing and uncertainty retention, adding association review/permission/audit tests, and completing an approved availability-provider integration with SSRF/network egress controls, monitoring, retention, and incident/rollback procedures.
 
+## Phase 8 website scanning boundary
+
+Website scanning is a bounded, tenant-scoped observation—not a crawler, browser, verifier, or outreach mechanism. A scan may fetch only `http` and `https` URLs after strict parsing and normalization. It must reject credentials, non-web schemes (`file:`, `ftp:`, `gopher:`, `data:`, `javascript:`, and similar), malformed hosts, localhost, IP literals where policy disallows them, and targets in loopback, private, link-local, multicast, reserved, or cloud-metadata ranges. DNS is resolved immediately before connection and the destination is revalidated at connection time; every redirect is limited, normalized, and revalidated for protocol, hostname, DNS, and IP range before it is followed. DNS answers must not be trusted from the initial validation alone (including rebinding changes).
+
+Each scan enforces hard budgets: total wall-clock/request time, response bytes, body bytes retained, redirect count, and page/link crawl count and depth. Budgets apply across redirects and discovered links, with bounded concurrency, retries, and response decompression; a limit, timeout, DNS error, unsupported content type, or partial fetch produces an explicit incomplete/unknown outcome rather than an empty result. The scanner fetches HTML and other explicitly allowed small resources only; it does not submit forms, send credentials, execute JavaScript, load browser plugins, or perform arbitrary subresource requests.
+
+Classifications are conservative and explainable. `unknown`, `blocked`, `timeout`, `partial`, and `error` remain distinct from a positive observation. A page can be classified only from bounded fetched content and must retain URL, redirect chain, response metadata, observed time, scanner/policy version, limits, and uncertainty reasons. A detected contact form, script, tracking tag, or business phrase is an observation—not proof of ownership, consent, deliverability, safety, or permission to contact.
+
+Scan history is tenant-scoped and append-oriented. Results and cache entries are keyed by normalized URL plus scanner/policy/version inputs, bounded by size and retention, and expose `observed_at`, freshness/expiry, and whether a result came from cache. A cache hit is never represented as a fresh scan; policy, DNS, or scanner-version changes require revalidation/invalidation. History must not leak response bodies, secrets, cookies, authorization headers, or unnecessary personal data across tenants.
+
+The website scanner remains a pilot boundary. Compose does not provide a production egress proxy, durable scan queue, distributed crawl coordinator, hardened DNS resolver, or compliance-grade result store. Production still requires independent SSRF testing (including DNS rebinding and redirect chains), egress/network policy, resource isolation, durable retention/deletion, authenticated scan-history authorization, rate limits and abuse controls, observability, and a reviewed policy for content types, robots/terms, caching, and incident response. Scans must never trigger acquisition, verification, enrichment, or outreach automatically.
+
 ## Verification
 
 ```bash