add conservative domain intelligence

docs/OPERATIONS.md

@@ -51,6 +51,16 @@ After a merge, read back the survivor and audit trail, check child/evidence/sour
 
 The current Compose/MVP runtime exposes the Phase 6 match-suggestion, explicit merge-confirmation, tenant-scoped merge-history, and reversal routes; no separate worker is required. It has basic canonicalization and exact duplicate handling plus pilot-grade fuzzy review. Monitor merge audit events and verify child/source-record counts after each operation. Do not schedule or report deduplication as automated discovery, enrichment, verification, or outreach.
 
+## Phase 7 domain-intelligence operations
+
+Phase 7 is a review contract only; the current Compose stack has no DNS worker, PSL service, DNS cache, or availability provider. Do not add a resolver or provider by pointing a job at an arbitrary URL. Any future implementation must record a pinned PSL version, preserve raw and registrable-domain forms, and report unresolved reasons for public suffixes, unknown/private suffixes, malformed/single-label values, localhost, and IP literals. IDN/punycode handling and subdomain treatment must be deterministic.
+
+Operate DNS as bounded, observable evidence collection. Surface `not_checked`, `pending`, `resolved`, `nxdomain`, `no_data`, `timeout`, `servfail`, `blocked`, and `error`; never translate an empty/error result into a business conclusion. Keep MX, NS, and TXT records separate and record resolver/source, observed time, TTL, freshness/staleness, truncation/partial state, and uncertainty/error reason. A cached answer is not a fresh check: caches must be bounded, tenant-isolated, keyed by normalized query/type/class and resolver policy, honor TTL, and be invalidated/re-evaluated when the PSL or resolver policy changes.
+
+Review domain-to-business candidates separately from DNS results. Verify tenant scope before comparison, reject public-suffix-only and malformed/IP candidates, and flag shared, parked, wildcard, homograph, sibling-subdomain, and conflicting-evidence cases. Association confidence must show evidence, reasons, algorithm/version, freshness, and uncertainty; it is suggestion-only. Require an authorized human accept/reject decision and audit it. Never auto-attach a domain or treat DNS as proof of ownership.
+
+Availability is `unknown` unless the API reports a result from an authorized provider. Before enabling one, verify current product/legal/security approval, terms owner, allowed tenant scope, request/concurrency and timeout limits, retention/deletion class, health/circuit state, and explicit operational enablement. `nxdomain`, `no_data`, timeout, stale cache, or provider error is not “available.” On provider outage, rate-limit, terms/approval expiry, circuit-open, or conflicting result, fail closed and report unknown/deferred; never purchase, reserve, contact, or retry through an unapproved provider.
+
 ## Phase 4 jobs and live logging
 
 The Phase 4 MVP provides SQLite-backed job status/detail/event routes and a browser monitor. A job moves `queued` → `running` → `succeeded`/`failed`/`cancelled`, retains its attempt and tenant identity, and appends per-job events with a monotonic sequence cursor. Operators inspect status and replay events by polling; SSE may provide lower-latency delivery but is not implemented and must replay from the persisted cursor and fall back to polling after disconnects.
@@ -107,7 +117,7 @@ Do not run `docker compose down -v` on a data-bearing environment: it removes th
 
 ## Production migration and scaling path
 
-Before production, complete a migration from SQLite to a reviewed production database, add schema/indexes for jobs/idempotency/events, implement transactional sequence assignment and tenant authorization, and prove cancellation/retry/lease recovery under concurrency. Add durable queue/worker operations, metrics and alerts for queue age, failures, retries, cancellation latency, event lag/gaps, and SSE connections; define backup/restore and event-retention drills. Redis, Celery, Postgres, schedulers, discovery adapters, and scanners are possible future components—not implicit Compose dependencies and not implemented by this MVP. No automated discovery or outreach may be inferred from the scaling path.
+Before production, complete a migration from SQLite to a reviewed production database, add schema/indexes for jobs/idempotency/events and domain observations, implement transactional sequence assignment and tenant authorization, and prove cancellation/retry/lease recovery under concurrency. Add durable queue/worker operations, bounded DNS/PSL processing, TTL-aware cache invalidation, uncertainty and association-review workflows, and a separately approved availability provider. Add metrics and alerts for queue age, failures, retries, cancellation latency, event lag/gaps, DNS status/error rates, cache freshness, provider rate limits/circuit state, and SSE connections; define backup/restore and event-retention drills. Redis, Celery, Postgres, schedulers, discovery adapters, and scanners are possible future components—not implicit Compose dependencies and not implemented by this MVP. No automated discovery, domain acquisition, ownership assertion, or outreach may be inferred from the scaling path.
 
 ## Incident checklist
 

docs/SECURITY.md

@@ -29,6 +29,17 @@
 
 The MVP provides deterministic match suggestions, an explicit human confirmation dialog, a tenant-scoped merge ledger/snapshot, and reversal routes. These are pilot controls, not a compliance-grade merge system: add a dedicated merge permission, server-verifiable confirmation/re-authentication, complete parent/conflict snapshots, transactional concurrency protection, and tamper-evident retention before production. Do not present derived normalization or a suggestion as verified identity, discovery, enrichment, or outreach permission.
 
+## Phase 7 domain-intelligence controls
+
+- Registrable-domain derivation must use a pinned/versioned PSL. Never use a naive last-two-label fallback. Preserve the raw input and return unresolved for unknown/private suffixes, public suffixes, malformed/single-label names, localhost, and IP literals; handle IDN/punycode deterministically and treat subdomains conservatively.
+- DNS status is not a business fact. Expose `not_checked`, `pending`, `resolved`, `nxdomain`, `no_data`, `timeout`, `servfail`, `blocked`, and `error` separately. A failed, blocked, stale, or empty response must not be upgraded to “no domain,” “no email,” or “unowned.”
+- MX, NS, and TXT observations require independent provenance, resolver/source, observed time, TTL, truncation/partial and uncertainty/error metadata. MX absence does not prove mail inability; NS does not prove control; TXT does not prove ownership. Treat TXT values as untrusted input and redact secrets where applicable.
+- DNS caches must be bounded and keyed by normalized name/type/class plus resolver policy and PSL version where relevant. Honor received TTLs without extending authority, expose freshness/staleness, invalidate on policy/PSL changes, and prevent one tenant's cached evidence from being disclosed to another. A cache hit is not a fresh check.
+- Association confidence is an explainable, versioned review signal separate from DNS status and duplicate score. Candidate generation must enforce tenant scope before comparison, reject public-suffix-only/malformed/IP candidates, flag shared/parked/wildcard/homograph/sibling-subdomain and conflicting-evidence cases, and require authorized human accept/reject decisions. Never auto-attach or infer ownership.
+- Domain availability must remain `unknown` unless an explicitly authorized provider is registered and checked at execution time for current product/legal/security approval, terms, tenant scope, rate/concurrency limits, retention, and health/circuit state. DNS `nxdomain`/`no_data`, stale cache, timeout, or provider failure is never availability. Fail closed; do not purchase, reserve, contact, or report availability from DNS alone.
+
+No Phase 7 resolver, cache, or availability provider is enabled in the current Compose runtime. Before production, add egress/SSRF controls, provider and PSL update review, retention/deletion handling, monitoring, permission/audit coverage, and failure/rollback tests for all domain observations.
+
 ## Phase 5 source security controls
 
 Source adapters are a security boundary, not a generic fetch facility. Registry review must verify the source identity, terms/robots and licensing owner, permitted collection purpose, approval expiry, tenant scope, rate/concurrency budget, raw-record retention/deletion policy, and circuit thresholds. Keep these controls server-side and auditable; a UI flag or client-supplied source ID is not authorization.
@@ -57,6 +68,7 @@ If a future approved adapter fetches URLs, apply the SSRF requirements below in
 11. **Jobs and idempotency:** require tenant-scoped idempotency keys for side-effecting job creation and bind each key to a request fingerprint; reject conflicting reuse and make retries safe against duplicate side effects. Persist lifecycle transitions transactionally and define lease/timeout/recovery behavior.
 12. **Live delivery:** SSE, if introduced, must authenticate before opening the stream, enforce tenant scope on every replay query, bound event/backlog size, support `Last-Event-ID`/cursor replay, send heartbeats, and provide polling fallback. Treat event-stream connections as untrusted clients and avoid cross-tenant timing/detail leaks.
 13. **Worker boundary:** the current SQLite/in-process MVP is not durable or horizontally safe. A production worker migration requires reviewed queue semantics, leases, visibility timeouts, dead-letter handling, concurrency limits, cancellation races, metrics, and deployment isolation. Redis/Celery are not implemented today.
+14. **Domain intelligence:** implement and security-review PSL pin/update handling, bounded DNS resolution, TTL-aware cache isolation/invalidation, uncertainty-preserving MX/NS/TXT parsing, tenant-scoped association decisions, and an authorized availability provider before exposing any live domain status. No DNS response may authorize acquisition, ownership, outreach, or verification.
 
 ## Source and contact policy