- ${reasonsText(item).map(reason=>`
- ${esc(typeof reason==='string'?reason:reason?.text||reason?.description||JSON.stringify(reason))} `).join('')||'
- Reason unavailable '}
diff --git a/README.md b/README.md index 4227039..7653261 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Prospect Intelligence Platform -A safety-first Phase 6 design/implementation boundary for **manual**, evidence-led prospect qualification and controlled source ingestion. The current runtime remains a manual vertical slice: it stores tenant-owned businesses and child intelligence records, keeps provenance with each evidence item, supports a review pipeline, and records operationally relevant changes. Phase 6 defines deterministic South African normalization and deduplication review semantics in addition to the Phase 5 source controls; it does **not** enable network discovery. **Automated outreach is disabled, and no live source may be enabled without explicit approval.** +A safety-first Phase 7 design/implementation boundary for **manual**, evidence-led prospect qualification, controlled source ingestion, and domain intelligence review. The current runtime remains a manual vertical slice: it stores tenant-owned businesses and child intelligence records, keeps provenance with each evidence item, supports a review pipeline, and records operationally relevant changes. Phase 7 defines conservative registrable-domain/PSL and DNS observation semantics in addition to the Phase 6 normalization and deduplication rules; it does **not** enable network discovery. **Automated outreach is disabled, and no live source may be enabled without explicit approval.** ## Included @@ -96,6 +96,18 @@ Every confirmed merge must create a tenant-scoped, immutable-enough merge snapsh The MVP now exposes deterministic match suggestions at `GET /api/v1/businesses/{id}/matches`, explicit merge confirmation in the web review dialog, tenant-scoped merge history, and `POST /api/v1/merge-history/{id}/reverse`. The implementation remains a pilot boundary: hardening is still needed for a dedicated merge permission, stronger server-side confirmation semantics, full snapshot conflict handling, and production-grade rollback guarantees. Do not describe a normalized or suggested match as verified identity, discovery, enrichment, or outreach authorization. +## Phase 7 domain intelligence boundary + +Domain intelligence is an observation and review aid, not proof of business identity, control of a domain, or availability. A domain normalizer may derive a lowercase ASCII/Unicode comparison form and a **registrable domain** using a versioned Public Suffix List (PSL). The PSL is an input with update/version drift: unknown, private, malformed, single-label, localhost, and IP-literal values must remain unresolved rather than guessed. A subdomain is not automatically a separate candidate, and a public suffix itself is never a registrable domain. + +DNS status is explicit: `not_checked`, `pending`, `resolved`, `nxdomain`, `no_data`, `timeout`, `servfail`, `blocked`, and `error` describe the check outcome, not a business conclusion. MX, NS, and TXT observations may be absent, partial, truncated, stale, resolver-dependent, or blocked; no record is not proof that mail, delegation, ownership, or a business relationship is absent. Store the resolver/source, observed time, TTL where supplied, and uncertainty/error metadata. Caches must be bounded and keyed by normalized query/type/class plus resolver policy, honor an observed TTL without extending authority, and expose freshness/staleness; cached data must never be presented as a fresh check. + +Association confidence is separate from DNS status and from duplicate score. It must be derived from explainable, tenant-scoped evidence (for example, an operator citation, an exact business-domain observation, or corroborating DNS facts), retain the algorithm/version and uncertainty reasons, and remain suggestion-only. Candidate generation must reject cross-tenant records, public-suffix-only values, malformed or IP-only inputs, and suppressed/merged targets as applicable; it must not auto-attach a domain or infer ownership from a shared, parked, wildcard, sibling-subdomain, homograph, or merely resolvable domain. Every candidate needs human review, provenance, and an auditable accept/reject decision. + +The platform must not claim that a domain is available, unregistered, or safe to acquire without an explicitly authorized availability provider registered with current product/legal/security approval, terms, tenant scope, rate limits, retention, and operational enablement. DNS `nxdomain` or `no_data` is not an availability result. Provider outages, rate limits, stale responses, conflicting results, and unknown status must remain `unknown`/`unavailable`, fail closed, and never trigger purchase, outreach, or automated follow-up. + +Phase 7 remains a documentation/contract boundary in this MVP: there is no live DNS resolver, PSL-backed enrichment worker, cache service, or availability provider in Compose. Production work still includes selecting and versioning the PSL, implementing bounded DNS resolution and TTL-aware cache invalidation, defining MX/NS/TXT parsing and uncertainty retention, adding association review/permission/audit tests, and completing an approved availability-provider integration with SSRF/network egress controls, monitoring, retention, and incident/rollback procedures. + ## Verification ```bash diff --git a/apps/api/README.md b/apps/api/README.md index 487b157..c06a396 100644 --- a/apps/api/README.md +++ b/apps/api/README.md @@ -1,6 +1,6 @@ -# Prospect Platform API — Phase 6 boundary +# Prospect Platform API — Phase 7 boundary -Dependency-light JSON API for tenant-scoped, **manual** prospect workflows and the Phase 6 normalization/deduplication plus Phase 5 source-ingestion contracts. Core domain rules use Python's standard library and persistence is SQLite. The API stores businesses plus child intelligence/evidence records, pipeline state, notes, and audit context. Source queries and adapter results must remain auditable and fail closed; the current runtime does not perform network discovery, DNS/website scanning, or outreach. +Dependency-light JSON API for tenant-scoped, **manual** prospect workflows and the Phase 7 domain-intelligence, Phase 6 normalization/deduplication, and Phase 5 source-ingestion contracts. Core domain rules use Python's standard library and persistence is SQLite. The API stores businesses plus child intelligence/evidence records, pipeline state, notes, and audit context. Source queries and adapter results must remain auditable and fail closed; the current runtime does not perform network discovery, DNS/website scanning, or outreach. ## Run @@ -18,6 +18,8 @@ Set `PROSPECT_API_PORT` or pass `--port`; set `PROSPECT_API_DB` or pass `--db` t All protected endpoints require the server-side session cookie. Every query is constrained by the authenticated user's `organization_id`; IDs from another tenant behave as not found and must not disclose whether a record exists. +Phase 7 domain routes (all tenant-scoped) are `POST /api/v1/businesses/{id}/domains/check`, `GET /api/v1/businesses/{id}/domains/check?domain=...`, `GET /api/v1/domain-checks`, `GET /api/v1/businesses/{id}/domain-candidates`, and `POST /api/v1/businesses/{id}/domain-candidates/check-availability`. The current implementation is intentionally conservative: a successful address lookup is reported as `ok`, unresolved/empty results as `unknown`, and an availability check returns `unknown`/`not_configured` because no provider is enabled. Treat these as observation states, not ownership or availability claims. + ## Phase 4 jobs and live logging (target contract) The intended job resource has a stable `job_id`, tenant/creator metadata, operation/payload fingerprint, `status`, `attempt`, timestamps, cancellation state, and terminal error/result metadata. Its lifecycle is `queued` → `running` → exactly one terminal state: `succeeded`, `failed`, or `cancelled`. State transitions and worker messages must be persisted transactionally with tenant and job identifiers; terminal jobs are immutable except for controlled retention/redaction. @@ -102,6 +104,18 @@ Remaining limitations: the snapshot currently focuses on the source graph rather List and child-record endpoints are deliberately bounded. For business lists, use `page` (starting at 1) and `page_size` within the server-enforced maximum; invalid values are rejected rather than allowing an unbounded query. Supported filters are applied inside the tenant-scoped query before pagination: `q`, `score_min`, `score_max`, `website_class`, and `pipeline_stage`. The UI's page and filter controls are convenience clients, not authorization controls. A filtered page is not a count of the entire unfiltered tenant unless the response explicitly says so. +## Phase 7 domain-intelligence contract + +Domain observations are tenant-scoped, provenance-bearing inputs. Registrable-domain normalization must use a pinned/versioned PSL rather than a last-two-label heuristic. Preserve the original value and normalized form; return `unknown` when the PSL cannot classify a value (including an unknown/private suffix, public suffix, malformed name, single-label name, localhost, or IP literal). IDN/punycode and case/label handling must be deterministic and must not turn a subdomain into an independent business identity. + +DNS checks must expose an explicit status—`not_checked`, `pending`, `resolved`, `nxdomain`, `no_data`, `timeout`, `servfail`, `blocked`, or `error`—and never collapse failure or absence into a negative business fact. MX, NS, and TXT are independently uncertain observations: retain record type, normalized response, resolver/source, observed time, TTL if supplied, truncation/partial indicators, and error/uncertainty reason. A missing MX does not prove that email is unavailable; an NS result does not prove control; TXT content does not prove ownership. + +Any DNS cache must be bounded, tenant-safe, keyed by normalized name/type/class and resolver policy, and TTL-aware. Do not extend authority beyond the received TTL; expose `observed_at`, `expires_at`/freshness, and stale or refresh state. A cache hit is not a fresh lookup, and resolver policy/PSL version changes require invalidation or re-evaluation. There is no DNS resolver or cache service in the current runtime. + +Association confidence is a separate, explainable, versioned review signal—not DNS status, duplicate score, or identity proof. Candidate-domain generation must apply the organization predicate before comparison, reject public-suffix-only/malformed/IP candidates, avoid automatic attachment, and flag shared/parked/wildcard/homograph/sibling-subdomain and conflicting-evidence cases. Human accept/reject decisions, reasons, provenance, and confidence version must be auditable; no candidate may authorize outreach or verification. + +Domain availability is `unknown` unless an explicitly authorized availability provider supplies it. The provider must be registered with current product/legal/security approval, terms owner, tenant scope, rate/concurrency limits, retention, health/circuit policy, and operational enablement checked at execution time. `nxdomain`, `no_data`, timeout, stale cache, or provider failure is not an availability result. Never purchase, reserve, contact, or report a domain as available from DNS alone. These safeguards are contract requirements only; no live availability provider is implemented. + ## Remaining limitations and production migration work -SQLite is a pilot store with an MVP job/event schema but no production migration runner, durable queue, scheduler, worker lease/recovery, durable backup command, or tested restore workflow. Authentication currently uses a development password fallback and does not by itself provide production Argon2id, MFA, CSRF protection, rate limiting, or a complete retention-grade audit system. Automated discovery, DNS/HTTP scanning, and outreach remain explicitly out of scope. Production migration work must add schema/index hardening and retention policy, tenant-scoped authorization tests, transactional event sequencing, cancellation/retry semantics, observability, and a reviewed Postgres plus durable queue/worker design. Redis/Celery remain future options, not implemented dependencies. +SQLite is a pilot store with an MVP job/event schema but no production migration runner, durable queue, scheduler, worker lease/recovery, durable backup command, or tested restore workflow. Authentication currently uses a development password fallback and does not by itself provide production Argon2id, MFA, CSRF protection, rate limiting, or a complete retention-grade audit system. Automated discovery, DNS/HTTP scanning, and outreach remain explicitly out of scope. Phase 7 production work must add a reviewed PSL update/version policy, bounded DNS workers, TTL-aware cache storage/invalidation, uncertainty-aware MX/NS/TXT schema and tests, association candidate safeguards and permissions, and an authorized availability provider with egress/rate/circuit/retention controls. It must also add schema/index hardening and retention policy, tenant-scoped authorization tests, transactional event sequencing, cancellation/retry semantics, observability, and a reviewed Postgres plus durable queue/worker design. Redis/Celery remain future options, not implemented dependencies. diff --git a/apps/api/app/domain_intelligence.py b/apps/api/app/domain_intelligence.py new file mode 100644 index 0000000..f0a5c71 --- /dev/null +++ b/apps/api/app/domain_intelligence.py @@ -0,0 +1,174 @@ +"""Conservative, dependency-free domain intelligence helpers. + +This module deliberately reports uncertainty rather than inferring DNS or registrar facts. +""" +from __future__ import annotations + +import ipaddress +import re +import socket +import threading +import time +import unicodedata +from datetime import datetime, timezone +from difflib import SequenceMatcher +from urllib.parse import urlparse + +# Small explicitly maintained PSL subset. Unknown suffixes are never guessed. +PUBLIC_SUFFIXES = frozenset({ + "com", "org", "net", "za", "co.za", "org.za", "net.za", "ac.za", "gov.za", "edu.za", +}) +DEFAULT_TTL_SECONDS = 300 +MAX_DOMAIN_LENGTH = 253 +MAX_CANDIDATES = 20 + + +def _host(value: str | None) -> str: + raw = str(value or "").strip().lower() + if not raw: + return "" + parsed = urlparse(raw if "://" in raw else "//" + raw) + host = (parsed.hostname or "").rstrip(".").lower() + if host.startswith("www."): + host = host[4:] + return host + + +def _valid_hostname(host: str) -> bool: + if not host or len(host) > MAX_DOMAIN_LENGTH or "." not in host: + return False + try: + ipaddress.ip_address(host) + return False + except ValueError: + pass + if any(len(label) > 63 or not re.fullmatch(r"[a-z0-9](?:[a-z0-9-]*[a-z0-9])?", label) for label in host.split(".")): + return False + return True + + +def normalize_registrable_domain(value: str | None) -> str: + """Return the registrable domain, or ``unknown`` for unsupported/unsafe input.""" + host = _host(value) + if not _valid_hostname(host): + return "unknown" + labels = host.split(".") + suffix = next((".".join(labels[-n:]) for n in (3, 2, 1) if ".".join(labels[-n:]) in PUBLIC_SUFFIXES), None) + if not suffix or len(labels) <= suffix.count(".") + 1: + return "unknown" + return ".".join(labels[-(suffix.count(".") + 2):]) + + +def domain_info(value: str | None) -> dict: + host = _host(value) + registered = normalize_registrable_domain(value) + return {"input": str(value or ""), "hostname": host, "registrable_domain": registered, + "status": "ok" if registered != "unknown" else "unknown"} + + +def _metadata(ttl_seconds: int = DEFAULT_TTL_SECONDS) -> dict: + now = time.time() + checked = datetime.fromtimestamp(now, timezone.utc).replace(microsecond=0).isoformat() + return {"checked_at": checked, "ttl_seconds": ttl_seconds, + "cache_expires_at": datetime.fromtimestamp(now + ttl_seconds, timezone.utc).replace(microsecond=0).isoformat()} + + +def resolve_domain(domain: str, *, timeout: float = 3.0, resolver=None, ttl_seconds: int = DEFAULT_TTL_SECONDS) -> dict: + """Resolve only a validated hostname using stdlib socket; never fetches a URL.""" + normalized = normalize_registrable_domain(domain) + host = _host(domain) + result = {"domain": normalized, "hostname": host, "status": "unknown", "addresses": [], **_metadata(ttl_seconds)} + if normalized == "unknown" or not _valid_hostname(host): + return result + resolver = resolver or socket.getaddrinfo + try: + # getaddrinfo has no per-call timeout. Run it in a daemon worker so a + # resolver stall cannot block an HTTP handler indefinitely. + outcome = {} + def lookup(): + try: outcome["records"] = resolver(host, None, 0, socket.SOCK_STREAM) + except BaseException as exc: outcome["exception"] = exc + worker = threading.Thread(target=lookup, daemon=True) + worker.start(); worker.join(max(0.01, float(timeout))) + if worker.is_alive(): + result["status"] = "timeout" + return result + if "exception" in outcome: raise outcome["exception"] + records = outcome.get("records", []) + addresses = sorted({str(item[4][0]) for item in records if item and len(item) > 4 and item[0] in (socket.AF_INET, socket.AF_INET6)}) + result["addresses"] = addresses + result["status"] = "ok" if addresses else "unknown" + except socket.gaierror as exc: + code = getattr(exc, "errno", None) + if code is None and exc.args: code = exc.args[0] + result["status"] = "nxdomain" if code in (socket.EAI_NONAME, socket.EAI_NODATA, -2, -5) else "error" + result["error_code"] = code + except (TimeoutError, socket.timeout): + result["status"] = "timeout" + except OSError as exc: + result["status"] = "error"; result["error_code"] = exc.__class__.__name__ + return result + + +def capability_hook(domain: str, capability: str, *, resolver=None, ttl_seconds: int = DEFAULT_TTL_SECONDS) -> dict: + """MX/NS/TXT hook. No optional resolver means not_configured, not empty.""" + normalized = normalize_registrable_domain(domain) + base = {"domain": normalized, "capability": capability, "status": "unknown", "records": [], **_metadata(ttl_seconds)} + if normalized == "unknown": return base + if capability not in {"mx", "ns", "txt"}: base.update(status="error", error_code="unsupported_capability"); return base + if resolver is None: + base.update(status="not_configured", reason="resolver_not_configured") + return base + try: + records = resolver(normalized, capability) + base.update(status="ok" if records else "unknown", records=list(records or [])) + except TimeoutError: base["status"] = "timeout" + except socket.gaierror: base["status"] = "nxdomain" + except Exception as exc: base.update(status="error", error_code=exc.__class__.__name__) + return base + + +def resolve_mx(domain: str, *, resolver=None, ttl_seconds: int = DEFAULT_TTL_SECONDS) -> dict: + return capability_hook(domain, "mx", resolver=resolver, ttl_seconds=ttl_seconds) + + +def resolve_ns(domain: str, *, resolver=None, ttl_seconds: int = DEFAULT_TTL_SECONDS) -> dict: + return capability_hook(domain, "ns", resolver=resolver, ttl_seconds=ttl_seconds) + + +def resolve_txt(domain: str, *, resolver=None, ttl_seconds: int = DEFAULT_TTL_SECONDS) -> dict: + return capability_hook(domain, "txt", resolver=resolver, ttl_seconds=ttl_seconds) + + +registrable_domain = normalize_registrable_domain + + +def association_confidence(candidate_domain: str, observed_domain: str, *, business_name: str = "") -> dict: + candidate = normalize_registrable_domain(candidate_domain); observed = normalize_registrable_domain(observed_domain) + if candidate == "unknown" or observed == "unknown": return {"level":"unknown", "score":0.0, "reasons":["unsupported_domain"]} + if candidate == observed: return {"level":"high", "score":1.0, "reasons":["same_registrable_domain"]} + token = re.sub(r"[^a-z0-9]", "", unicodedata.normalize("NFKD", str(business_name).lower())) + score = SequenceMatcher(None, token, candidate.split(".")[0]).ratio() if token else 0.0 + return {"level":"medium" if score >= .8 else "low", "score":round(score, 4), "reasons":["name_domain_similarity"] if score >= .8 else ["different_registrable_domain"]} + + +def _slug(value: object) -> str: + text = unicodedata.normalize("NFKD", str(value or "")).encode("ascii", "ignore").decode().lower() + words = re.findall(r"[a-z0-9]+", text) + return "-".join(words)[:40].strip("-") + + +def generate_candidate_domains(business_name: str, service: str = "", location: str = "", *, suffixes=("co.za", "com"), limit: int = MAX_CANDIDATES) -> list[str]: + limit = max(0, min(int(limit), MAX_CANDIDATES)); name, svc, loc = _slug(business_name), _slug(service), _slug(location) + if not name: return [] + parts = [name, f"{name}-{svc}" if svc else "", f"{name}-{loc}" if loc else "", f"{svc}-{loc}" if svc and loc else ""] + if svc: parts += [f"{name}{svc}"] + out=[] + for base in parts: + if not base or len(base) > 63: continue + for suffix in suffixes: + if suffix not in PUBLIC_SUFFIXES: continue + domain = f"{base}.{suffix}" + if normalize_registrable_domain(domain) != "unknown" and domain not in out: out.append(domain) + if len(out) >= limit: return out + return out diff --git a/apps/api/app/main.py b/apps/api/app/main.py index 32b1b7e..da6adc3 100644 --- a/apps/api/app/main.py +++ b/apps/api/app/main.py @@ -9,15 +9,17 @@ if __package__ in (None, ""): sys.path.insert(0, str(Path(__file__).resolve().parents[1])) from app.domain import deduplication_key, deduplicate_businesses, is_suppressed, normalize_business, score_business, normalize_domain, normalize_phone, match_businesses from app.sources import adapter_for, contains_secret + from app.domain_intelligence import normalize_registrable_domain, resolve_domain, generate_candidate_domains else: from .domain import deduplication_key, deduplicate_businesses, is_suppressed, normalize_business, score_business, normalize_domain, normalize_phone, match_businesses from .sources import adapter_for, contains_secret + from .domain_intelligence import normalize_registrable_domain, resolve_domain, generate_candidate_domains ORGANIZATION_ID = "demo-tenant" SCHEMA = Path(__file__).resolve().parents[1] / "schema.sql" SESSION_DAYS = 7 PBKDF2_ITERATIONS = 300_000 MUTATING_ROLES = {"owner", "admin", "researcher"} -JOB_TYPES = {"noop", "prospect_recalculate", "source_discovery"} +JOB_TYPES = {"noop", "prospect_recalculate", "source_discovery", "domain_check"} JOB_PAGE_SIZE = 100 SECRET_KEYS = {"password", "passwd", "secret", "token", "api_key", "apikey", "authorization", "credential", "private_key"} CHILD_TABLES = {"contacts": ("name", "email", "phone", "title", "do_not_contact"), "domains": ("domain", "kind"), "websites": ("url", "website_class"), "evidence": ("kind", "url", "claim"), "notes": ("body",)} @@ -117,16 +119,79 @@ class ApiHandler(BaseHTTPRequestHandler): if path=="/api/v1/discovery-queries": return self.list_queries(db,org) if path=="/api/v1/source-records": return self.list_source_records(db,org,parse_qs(parsed.query)) if path=="/api/v1/jobs": return self.list_jobs(db,org,parse_qs(parsed.query)) + if path=="/api/v1/domain-checks": return self.list_domain_checks(db,org,parse_qs(parsed.query)) if path.startswith("/api/v1/jobs/"): return self.get_job_route(db,org,path,parse_qs(parsed.query)) if path.startswith("/api/v1/businesses/"): bits=path.split("/"); ident=bits[4] if len(bits)>4 else "" if not ident.isdigit(): return self.send_json(404,{"error":"not_found"}) row=self.business(db,int(ident),org) if not row:return self.send_json(404,{"error":"not_found"}) + if len(bits)==7 and bits[5:]==["domains","check"]: return self.get_domain_check(int(ident),db,user,parse_qs(parsed.query)) + if len(bits)==6 and bits[5]=="domain-candidates": return self.list_domain_candidates(int(ident),db,org) if len(bits)==6 and bits[5]=="matches": return self.matches(int(ident),db,org) payload=row_json(row); payload.update(self.nested(db,int(ident),org)); return self.send_json(200,payload) return self.send_json(404,{"error":"not_found"}) finally: db.close() + def _domain_result(self, row, cache_hit=False): + try: result=json.loads(row["result_json"] or "{}") + except (TypeError,ValueError): result={} + result.update({"id":row["id"],"business_id":row["business_id"],"domain":row["domain"],"status":row["status"],"cache_hit":cache_hit,"checked_at":row["checked_at"],"cache_expires_at":row["cache_expires_at"]}) + return result + + def _check_domain(self, bid, domain, db, user): + normalized=normalize_registrable_domain(domain) + if normalized == "unknown": return self.send_json(400,{"error":"unsupported_domain","status":"unknown"}) + now=datetime.now(timezone.utc).replace(microsecond=0).isoformat() + cache_key=normalized+":a_aaaa:v1" + cached=db.execute("SELECT * FROM domain_checks WHERE organization_id=? AND business_id=? AND cache_key=? AND cache_expires_at>? ORDER BY id DESC LIMIT 1",(user["organization_id"],bid,cache_key,now)).fetchone() + if cached:return self.send_json(200,self._domain_result(cached,True)) + result=resolve_domain(normalized) + try: cur=db.execute("INSERT INTO domain_checks(organization_id,business_id,domain,status,result_json,cache_key,checked_at,cache_expires_at) VALUES(?,?,?,?,?,?,?,?)",(user["organization_id"],bid,normalized,result["status"],json.dumps(result,sort_keys=True),cache_key,result["checked_at"],result["cache_expires_at"])) + except sqlite3.IntegrityError: + existing=db.execute("SELECT id FROM domain_checks WHERE organization_id=? AND business_id=? AND cache_key=?",(user["organization_id"],bid,cache_key)).fetchone() + if not existing: return self.send_json(409,{"error":"domain_check_conflict"}) + db.execute("UPDATE domain_checks SET domain=?,status=?,result_json=?,checked_at=?,cache_expires_at=? WHERE id=?",(normalized,result["status"],json.dumps(result,sort_keys=True),result["checked_at"],result["cache_expires_at"],existing["id"])) + self.audit(db,user,"domain.checked",f"{bid}:{normalized}:{result['status']}"); db.commit() + return self.send_json(200,self._domain_result(db.execute("SELECT * FROM domain_checks WHERE id=?",(existing["id"],)).fetchone())) + self.audit(db,user,"domain.checked",f"{bid}:{normalized}:{result['status']}"); db.commit() + return self.send_json(200,self._domain_result(db.execute("SELECT * FROM domain_checks WHERE id=?",(cur.lastrowid,)).fetchone())) + + def post_domain_check(self,bid,payload,db,user): + if not self.business(db,bid,user["organization_id"]): return self.send_json(404,{"error":"not_found"}) + domain=payload.get("domain") or self.business(db,bid,user["organization_id"])["website_domain"] + if not str(domain).strip(): return self.send_json(400,{"error":"domain_required"}) + return self._check_domain(bid,str(domain),db,user) + + def get_domain_check(self,bid,db,user,query): + domain=(query.get("domain") or [""])[0] + if not domain:return self.send_json(400,{"error":"domain_required"}) + return self._check_domain(bid,domain,db,user) + + def list_domain_checks(self,db,org,query): + try: limit=max(1,min(int((query.get("page_size") or [50])[0]),100)); offset=max(0,int((query.get("offset") or [0])[0])) + except (ValueError,TypeError): return self.send_json(400,{"error":"invalid_pagination"}) + rows=db.execute("SELECT * FROM domain_checks WHERE organization_id=? ORDER BY id DESC LIMIT ? OFFSET ?",(org,limit+1,offset)).fetchall() + return self.send_json(200,{"organization_id":org,"items":[self._domain_result(r) for r in rows[:limit]],"limit":limit,"offset":offset,"has_more":len(rows)>limit}) + + def list_domain_candidates(self,bid,db,org): + business=self.business(db,bid,org) + if not business:return self.send_json(404,{"error":"not_found"}) + rows=db.execute("SELECT * FROM domain_candidates WHERE organization_id=? AND business_id=? ORDER BY rank,id",(org,bid)).fetchall() + if not rows: + generated=generate_candidate_domains(business["name"],business["description"],business["city"]) + for rank,domain in enumerate(generated,1): + db.execute("INSERT OR IGNORE INTO domain_candidates(organization_id,business_id,domain,rank) VALUES(?,?,?,?)",(org,bid,domain,rank)) + db.commit(); rows=db.execute("SELECT * FROM domain_candidates WHERE organization_id=? AND business_id=? ORDER BY rank,id",(org,bid)).fetchall() + return self.send_json(200,{"business_id":bid,"items":[row_json(r) for r in rows]}) + + def check_availability(self,bid,payload,db,user): + if not self.business(db,bid,user["organization_id"]):return self.send_json(404,{"error":"not_found"}) + candidates=db.execute("SELECT domain FROM domain_candidates WHERE organization_id=? AND business_id=? ORDER BY rank,id",(user["organization_id"],bid)).fetchall() + domains=[r["domain"] for r in candidates] + if isinstance(payload.get("domains"),list): domains=[normalize_registrable_domain(x) for x in payload["domains"] if normalize_registrable_domain(x)!="unknown"][:20] + self.audit(db,user,"domain.availability.checked",str(bid)); db.commit() + return self.send_json(200,{"business_id":bid,"status":"unknown","reason":"not_configured","provider_configured":False,"items":[{"domain":d,"status":"unknown","reason":"not_configured"} for d in domains]}) + def list_jobs(self, db, org, query): try: limit=max(1,min(int(query.get("page_size",[50])[0]),JOB_PAGE_SIZE)); offset=max(0,int(query.get("offset",[0])[0])) except (ValueError, TypeError): return self.send_json(400,{"error":"invalid_pagination"}) @@ -221,6 +286,8 @@ class ApiHandler(BaseHTTPRequestHandler): if path=="/api/v1/discovery-queries":return self.create_query(payload,db,user) if path=="/api/v1/suppressions":return self.create_suppression(payload,db,user) if path=="/api/v1/imports/preview":return self.preview_import(payload,db,org) + if len(path.split("/"))==7 and path.split("/")[3:6]==["businesses",path.split("/")[4],"domains"] and path.split("/")[6]=="check": return self.post_domain_check(int(path.split("/")[4]) if path.split("/")[4].isdigit() else -1,payload,db,user) + if len(path.split("/"))==7 and path.split("/")[3:6]==["businesses",path.split("/")[4],"domain-candidates"] and path.split("/")[6]=="check-availability": return self.check_availability(int(path.split("/")[4]) if path.split("/")[4].isdigit() else -1,payload,db,user) if path.startswith("/api/v1/merge-history/") and path.endswith("/reverse"): ident=path.split("/")[4] return self.reverse_merge(int(ident) if ident.isdigit() else -1,db,user) diff --git a/apps/api/schema.sql b/apps/api/schema.sql index c4276a2..11d4763 100644 --- a/apps/api/schema.sql +++ b/apps/api/schema.sql @@ -153,3 +153,20 @@ CREATE TABLE IF NOT EXISTS source_records ( UNIQUE(organization_id,source_id,content_hash) ); CREATE INDEX IF NOT EXISTS idx_source_records_org ON source_records(organization_id,id DESC); + +-- Phase 7 domain intelligence (additive-safe; DNS results are explicitly stateful). +CREATE TABLE IF NOT EXISTS domain_checks ( + id INTEGER PRIMARY KEY AUTOINCREMENT, organization_id TEXT NOT NULL REFERENCES organizations(id), + business_id INTEGER NOT NULL REFERENCES businesses(id) ON DELETE CASCADE, domain TEXT NOT NULL, + status TEXT NOT NULL, result_json TEXT NOT NULL DEFAULT '{}', cache_key TEXT NOT NULL, + checked_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP, cache_expires_at TEXT, created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP, + UNIQUE(organization_id, business_id, cache_key) +); +CREATE INDEX IF NOT EXISTS idx_domain_checks_org ON domain_checks(organization_id,created_at DESC,id DESC); +CREATE INDEX IF NOT EXISTS idx_domain_checks_business ON domain_checks(organization_id,business_id,created_at DESC); +CREATE TABLE IF NOT EXISTS domain_candidates ( + id INTEGER PRIMARY KEY AUTOINCREMENT, organization_id TEXT NOT NULL REFERENCES organizations(id), business_id INTEGER NOT NULL REFERENCES businesses(id) ON DELETE CASCADE, + domain TEXT NOT NULL, source TEXT NOT NULL DEFAULT 'generated', rank INTEGER NOT NULL DEFAULT 0, metadata_json TEXT NOT NULL DEFAULT '{}', + created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP, UNIQUE(organization_id,business_id,domain) +); +CREATE INDEX IF NOT EXISTS idx_domain_candidates_business ON domain_candidates(organization_id,business_id,rank,id); diff --git a/apps/api/tests/test_phase7_domain.py b/apps/api/tests/test_phase7_domain.py new file mode 100644 index 0000000..5a2dd3f --- /dev/null +++ b/apps/api/tests/test_phase7_domain.py @@ -0,0 +1,92 @@ +import json +import os +import sqlite3 +import threading +import unittest +from http.client import HTTPConnection +from tempfile import TemporaryDirectory +from unittest.mock import patch + +from app.domain_intelligence import ( + normalize_registrable_domain, resolve_domain, generate_candidate_domains, + association_confidence, resolve_mx, +) +from app.main import create_server + + +class DomainIntelligenceTests(unittest.TestCase): + def test_psl_normalization_and_unknown_suffix(self): + self.assertEqual(normalize_registrable_domain('https://WWW.shop.example.co.za/path'), 'example.co.za') + self.assertEqual(normalize_registrable_domain('foo.example.com'), 'example.com') + self.assertEqual(normalize_registrable_domain('foo.example.invalidtld'), 'unknown') + + def test_candidate_generation_is_bounded_and_safe(self): + out = generate_candidate_domains('Acme & Sons (Pty) Ltd', 'Solar Panels', 'Cape Town') + self.assertLessEqual(len(out), 20) + self.assertTrue(out) + for value in out: + self.assertRegex(value, r'^[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?\.(?:co\.za|com)$') + self.assertNotIn('--', value) + + def test_resolution_distinguishes_nxdomain_timeout_and_error(self): + self.assertEqual(resolve_domain('missing.example.com', resolver=lambda *a: (_ for _ in ()).throw(socket_gaierror_name()))['status'], 'nxdomain') + self.assertEqual(resolve_domain('slow.example.com', resolver=lambda *a: (_ for _ in ()).throw(TimeoutError()))['status'], 'timeout') + self.assertEqual(resolve_domain('bad.example.com', resolver=lambda *a: (_ for _ in ()).throw(OSError('boom')))['status'], 'error') + + def test_local_resolution_and_association_confidence(self): + with patch('app.domain_intelligence.socket.getaddrinfo', return_value=[(2, 1, 6, '', ('1.2.3.4', 0))]): + result = resolve_domain('example.com') + self.assertEqual(result['status'], 'ok') + self.assertEqual(result['addresses'], ['1.2.3.4']) + self.assertEqual(association_confidence('acme.co.za', 'acme.co.za')['level'], 'high') + self.assertEqual(association_confidence('acme.co.za', 'other.co.za')['level'], 'low') + + def test_optional_capabilities_are_not_falsely_empty(self): + self.assertEqual(resolve_mx('example.com')['status'], 'not_configured') + + +def socket_gaierror_name(): + import socket + return socket.gaierror(socket.EAI_NONAME) + + +class DomainApiTests(unittest.TestCase): + def setUp(self): + self.tmp = TemporaryDirectory(); self.db_path = self.tmp.name + '/db.sqlite' + os.environ['BOOTSTRAP_ADMIN_EMAIL'] = 'domain-owner@example.test'; os.environ['BOOTSTRAP_ADMIN_PASSWORD'] = 'password' + self.server = create_server('127.0.0.1', 0, self.db_path); self.thread = threading.Thread(target=self.server.serve_forever, daemon=True); self.thread.start() + self.conn = HTTPConnection('127.0.0.1', self.server.server_port, timeout=4); self.cookie = None + self.request('POST', '/api/v1/auth/login', {'email':'domain-owner@example.test','password':'password'}) + + def tearDown(self): + self.server.shutdown(); self.server.server_close(); self.thread.join(2); self.tmp.cleanup() + + def request(self, method, path, payload=None): + body = json.dumps(payload).encode() if payload is not None else None; headers = {'Content-Type':'application/json'} if body else {} + if self.cookie: headers['Cookie'] = self.cookie + self.conn.request(method, path, body, headers); response = self.conn.getresponse(); c = response.getheader('Set-Cookie') + if c: self.cookie = c.split(';', 1)[0] + return response.status, json.loads(response.read() or b'{}') + + def test_check_cache_candidates_and_no_false_availability(self): + status, business = self.request('POST', '/api/v1/businesses', {'name':'Acme Solar','website':'https://example.com','city':'Cape Town','province':'Western Cape'}) + self.assertEqual(status, 201); bid = business['id'] + with patch('app.main.resolve_domain', return_value={'status':'unknown','addresses':[],'checked_at':'2026-01-01T00:00:00+00:00','cache_expires_at':'2099-01-01T00:00:00+00:00'}): + status, first = self.request('POST', f'/api/v1/businesses/{bid}/domains/check', {'domain':'example.com'}) + self.assertEqual(status, 200); self.assertIn(first['status'], ('unknown','ok')) + status, second = self.request('GET', f'/api/v1/businesses/{bid}/domains/check?domain=example.com') + self.assertEqual(status, 200); self.assertTrue(second.get('cache_hit')) + status, candidates = self.request('GET', f'/api/v1/businesses/{bid}/domain-candidates') + self.assertEqual(status, 200); self.assertTrue(candidates['items']) + status, availability = self.request('POST', f'/api/v1/businesses/{bid}/domain-candidates/check-availability', {}) + self.assertEqual(status, 200); self.assertEqual(availability['status'], 'unknown'); self.assertEqual(availability['reason'], 'not_configured') + self.assertEqual(self.request('GET', '/api/v1/domain-checks')[0], 200) + + def test_business_domain_checks_are_tenant_scoped(self): + _, business = self.request('POST', '/api/v1/businesses', {'name':'Private'}) + self.cookie = None + self.request('POST', '/api/v1/auth/logout') + self.assertEqual(self.request('GET', f'/api/v1/businesses/{business["id"]}/domains/check?domain=example.com')[0], 401) + + +if __name__ == '__main__': unittest.main() diff --git a/apps/web/README.md b/apps/web/README.md index 15203a9..c0e75c1 100644 --- a/apps/web/README.md +++ b/apps/web/README.md @@ -1,6 +1,6 @@ -# ProspectOS web — Phase 6 boundary +# ProspectOS web — Phase 7 boundary -Self-contained static frontend for the Prospect Platform API. There is no bundler or runtime dependency: serve this directory with any static HTTP server. The UI supports a manual, tenant-scoped review workflow and a Phase 4 MVP job monitor. Phase 5 source concepts and Phase 6 normalization/deduplication concepts are display/contract boundaries only; the UI does not perform network discovery, scan DNS/websites, or send outreach. +Self-contained static frontend for the Prospect Platform API. There is no bundler or runtime dependency: serve this directory with any static HTTP server. The UI supports a manual, tenant-scoped review workflow and a Phase 4 MVP job monitor. Phase 5 source concepts, Phase 6 normalization/deduplication, and Phase 7 domain-intelligence concepts are display/contract boundaries only; the UI does not perform network discovery, scan DNS/websites, or send outreach. ## Configure and run @@ -62,4 +62,4 @@ A zero-dependency static smoke page (`smoke-test.html`) checks the key DOM contr ## Remaining limitations -The static client has no background discovery, DNS/website scanner, enrichment scheduler, outreach integration, or SSE delivery. It cannot make missing provenance authoritative and should display API-provided limitations rather than infer them. CSV preview is capped for display and is not an import workflow. Production deployment still requires durable job execution, tenant-scoped controls, idempotency verification, SSE/polling verification, and the security and operations gates in `docs/SECURITY.md` and `docs/OPERATIONS.md`. +The static client has no background discovery, DNS/website scanner, enrichment scheduler, outreach integration, availability provider, or SSE delivery. For future domain intelligence, display registrable-domain/PSL version and unresolved reasons, DNS status and freshness/TTL (not “available”), independent MX/NS/TXT uncertainty, and association confidence with explainable evidence. Never auto-attach candidate domains or treat `nxdomain`/`no_data` as availability; any availability label requires an API result from an authorized provider and explicit human review. It cannot make missing provenance authoritative and should display API-provided limitations rather than infer them. CSV preview is capped for display and is not an import workflow. Production deployment still requires durable job execution, tenant-scoped controls, idempotency verification, PSL/DNS/cache implementation and tests, candidate review permissions/audit, authorized availability-provider controls, SSE/polling verification, and the security and operations gates in `docs/SECURITY.md` and `docs/OPERATIONS.md`. diff --git a/apps/web/app.js b/apps/web/app.js index c40132b..ab5e45b 100644 --- a/apps/web/app.js +++ b/apps/web/app.js @@ -12,7 +12,7 @@ const freshness = (p) => { const raw=p.updated_at||p.last_checked_at||p.created_at; if(!raw)return {label:'Unknown',cls:'stale'}; const days=Math.max(0,Math.floor((Date.now()-new Date(raw).getTime())/86400000)); return {label:days===0?'Today':`${days}d ago`,cls:days<=7?'good':'stale'}; }; const scoreFor = (p) => Number.isFinite(Number(p.score)) ? Number(p.score) : (p.score_factors||p.factors||[]).reduce((n,f)=>n+({named_business:20,business_site:30,email:25,phone:15,description:10}[f]||0),0); const labelFactor = (f) => ({named_business:'Named business',business_site:'Business site',email:'Email found',phone:'Phone found',description:'Description'}[f] || String(f).replaceAll('_',' ')); - async function request(path, options = {}) { const response=await fetch(endpoint(path),{...options,credentials:'include'}); if(response.status===401){showLogin('Your session has expired. Please sign in again.');throw new Error('unauthorized');} return response; } + async function request(path, options = {}) { const response=await fetch(endpoint(path),{...options,credentials:'include'}); if(response.status===401){showLogin('Your session has expired. Please sign in again.');throw new Error('unauthorized');} if(response.status===403)throw new Error('Tenant/workspace access denied.'); return response; } async function jsonRequest(path, options = {}) { const res=await request(path,options); const body=await res.json().catch(()=>({})); if(!res.ok)throw new Error(body.error||body.message||'Request failed'); return body; } function showLogin(message=''){currentUser=null;$('dashboardShell').hidden=true;$('loginScreen').hidden=false;$('loginMessage').textContent=message;$('loginMessage').className=`form-message${message?' error':''}`;} function showDashboard(user){currentUser=user||{};const name=currentUser.name||currentUser.full_name||currentUser.email||'Workspace member';const role=currentUser.role||currentUser.roles?.[0]||'Member';$('userIdentity').textContent=`${name} · ${role}`;$('userAvatar').textContent=name.split(/\s+/).map(x=>x[0]).join('').slice(0,2).toUpperCase();$('loginScreen').hidden=true;$('dashboardShell').hidden=false;updateJobPermissions();} @@ -25,7 +25,38 @@ async function selectProspect(id){selectedId=Number(id);selectedDetail=null;renderRows();$('detailPanel').innerHTML='
${esc(error.message)}
${empty}
`; - function renderDetail(p){const s=scoreFor(p),st=statusOf(p),f=freshness(p),factors=p.score_factors||p.factors||[],blocked=st==='review'||st==='suppressed',stage=p.pipeline_stage||p.pipeline?.stage||(Array.isArray(p.pipeline)?p.pipeline.at(-1)?.stage:'')||'new',contacts=p.contacts||[],domains=p.domains||[],websites=p.websites||[],evidence=p.evidence||p.evidence_timeline||[],notes=p.notes||[],review=p.review_status||p.review|| (st==='review'?'Needs review':st[0].toUpperCase()+st.slice(1));$('detailPanel').innerHTML=`PROSPECT DETAIL
${esc(p.website_domain||'no detected website')}
✓ ${esc(labelFactor(x))}${esc(p.confidence||'Medium')}
`).join(''):''}${esc(review)}
${st!=='suppressed'?'':''}${st==='suppressed'?'Suppressed records cannot be contacted.':'Review this prospect before outreach is available.'}
`:''}`;renderDedupPanel();} + const domainValue = item => typeof item === 'string' ? item : item?.domain || item?.hostname || item?.name || item?.value || ''; + const domainForProspect = p => p?.website_domain || (Array.isArray(p?.domains) ? domainValue(p.domains[0]) : '') || ''; + const domainState = value => { const state=String(value ?? '').toLowerCase().replaceAll('_','-'); return state==='nxdomain'?'NXDOMAIN':state==='error'?'Error':state==='not-configured'?'Not configured':state==='unknown'||!state?'Unknown':state.replaceAll('-',' '); }; + const capabilityState = value => { if (value === true || ['present','found','yes','true','ok'].includes(String(value).toLowerCase())) return 'Present'; if (value === false || ['absent','not-found','no','false'].includes(String(value).toLowerCase())) return 'Not found'; return domainState(value); }; + const checkedTime = item => item?.checked_at || item?.checkedAt || item?.last_checked_at || item?.checked_time || 'Time unavailable'; + const cacheState = item => item?.cache_hit === true ? 'Hit' : item?.cache_state || item?.cacheStatus || item?.cache || (item?.cache_hit === false ? 'Miss' : 'Unknown'); + const confidenceText = item => item?.confidence ?? item?.score ?? 'Unknown'; + const reasonsText = item => { const reasons=item?.reasons || item?.reason || item?.explanation || []; return Array.isArray(reasons) ? reasons : [reasons]; }; + function renderDomainPanel(p){ + const panel=$('domainIntelligencePanel'); if(!panel)return; + panel.innerHTML=`DOMAIN INTELLIGENCE
DNS results are evidence only. Unknown, error, and NXDOMAIN states are never treated as ownership or availability.
No DNS check has been returned. Run a check to see evidence.
';return;} + const dns=result.dns || result.dns_state || result.state || result.status || 'unknown', records=result.records || result.capabilities || result; + const capability=(key, aliases=[])=>{for(const k of [key,...aliases])if(records?.[k]!==undefined)return records[k];return undefined;}; + el.innerHTML=`${esc(result.error||result.error_code||result.message)}
`:''}`; + } + function renderCandidates(payload){ + const panel=$('candidateDomainPanel'); if(!panel)return; + const candidates=payloadItems(payload,['candidates','candidate_domains','domains','items']); + panel.innerHTML=`CANDIDATE DOMAINS
No candidate domains returned.
'}Availability is not configured here. This action will never claim a domain is available or owned.
`; + } + async function loadDomainIntelligence(id){ + const check=$('domainCheckState'), candidates=$('candidateDomainPanel'); + try{const [checkPayload,candidatePayload]=await Promise.all([jsonRequest(`/api/v1/businesses/${encodeURIComponent(id)}/domains/check?domain=${encodeURIComponent(domainForProspect(selectedDetail || prospects.find(item=>Number(item.id)===Number(id)) || {}))}`),jsonRequest(`/api/v1/businesses/${encodeURIComponent(id)}/domain-candidates`)]);renderDomainCheck(checkPayload);renderCandidates(candidatePayload);}catch(error){if(error.message!=='unauthorized'){if(check)check.innerHTML=`${esc(error.message)}
Candidate domains unavailable.
';}} + } + async function runDomainCheck(){const button=$('runDomainCheckBtn');if(!selectedId||!button)return;button.disabled=true;const state=$('domainCheckState');if(state)state.innerHTML='${esc(error.message)}
PROSPECT DETAIL
${esc(p.website_domain||'no detected website')}
✓ ${esc(labelFactor(x))}${esc(p.confidence||'Medium')}
`).join(''):''}${esc(review)}
${st!=='suppressed'?'':''}${st==='suppressed'?'Suppressed records cannot be contacted.':'Review this prospect before outreach is available.'}
`:''}`;renderDomainPanel(p);renderDedupPanel();} let mergeSource = null, mergeTarget = null, mergeBusy = false; const payloadItems = (payload, keys) => { for (const key of keys) if (Array.isArray(payload?.[key])) return payload[key]; return Array.isArray(payload) ? payload : []; }; const suggestionId = item => item.target_id ?? item.business_id ?? item.prospect_id ?? item.id; @@ -112,7 +143,7 @@ async function logout(){try{await fetch(endpoint('/api/v1/auth/logout'),{method:'POST',credentials:'include'});}finally{showLogin('You have been signed out.');$('loginForm').reset();}} async function bootstrap(){try{const res=await fetch(endpoint('/api/v1/auth/me'),{credentials:'include'});if(res.status===401){showLogin();return;}if(!res.ok)throw new Error('Could not verify session.');const user=await res.json();showDashboard(user.user||user);await loadData();await loadJobs();await loadSources();}catch(e){if(e.message!=='unauthorized')showLogin('Unable to connect to the workspace. Try again.');}} document.addEventListener('submit',e=>{if(e.target.id==='contactForm')saveContact(e.target);if(e.target.id==='noteForm')saveNote(e.target);if(e.target.id==='pipelineForm')saveStage(e.target);}); - document.addEventListener('click',e=>{if(e.target.id==='verifyBtn')verify();if(e.target.id==='retryDetailBtn'&&selectedId)loadDetail(selectedId);if(e.target.id==='retryDedupBtn'&&selectedId)loadMatchSuggestions(selectedId);if(e.target.id==='retryHistoryBtn'&&selectedId)loadMergeHistory(selectedId);if(e.target.id==='cancelMergeBtn'||e.target.id==='cancelMergeBtnSecondary')closeMergeDialog();if(e.target.id==='confirmMergeBtn')confirmMerge();const mergeButton=e.target.closest?.('[data-merge-target]');if(mergeButton)openMergeDialog(mergeButton.dataset.mergeTarget,mergeButton.dataset.mergeTargetName);const reverseButton=e.target.closest?.('[data-reverse-merge]');if(reverseButton)reverseMerge(reverseButton.dataset.reverseMerge);if(e.target.id==='retryJobDetailBtn'&&selectedJobId)loadJobDetail(selectedJobId);if(e.target.id==='cancelJobBtn')jobAction('cancel');if(e.target.id==='retryJobBtn')jobAction('retry');const row=e.target.closest?.('[data-job-id]');if(row)loadJobDetail(row.dataset.jobId);}); + document.addEventListener('click',e=>{if(e.target.id==='verifyBtn')verify();if(e.target.id==='runDomainCheckBtn')runDomainCheck();if(e.target.id==='retryDomainBtn'&&selectedId)loadDomainIntelligence(selectedId);const availability=e.target.closest?.('[data-domain-availability]');if(availability)checkDomainAvailability(availability.dataset.domain,availability);if(e.target.id==='retryDetailBtn'&&selectedId)loadDetail(selectedId);if(e.target.id==='retryDedupBtn'&&selectedId)loadMatchSuggestions(selectedId);if(e.target.id==='retryHistoryBtn'&&selectedId)loadMergeHistory(selectedId);if(e.target.id==='cancelMergeBtn'||e.target.id==='cancelMergeBtnSecondary')closeMergeDialog();if(e.target.id==='confirmMergeBtn')confirmMerge();const mergeButton=e.target.closest?.('[data-merge-target]');if(mergeButton)openMergeDialog(mergeButton.dataset.mergeTarget,mergeButton.dataset.mergeTargetName);const reverseButton=e.target.closest?.('[data-reverse-merge]');if(reverseButton)reverseMerge(reverseButton.dataset.reverseMerge);if(e.target.id==='retryJobDetailBtn'&&selectedJobId)loadJobDetail(selectedJobId);if(e.target.id==='cancelJobBtn')jobAction('cancel');if(e.target.id==='retryJobBtn')jobAction('retry');const row=e.target.closest?.('[data-job-id]');if(row)loadJobDetail(row.dataset.jobId);}); $('loginForm').addEventListener('submit',login);$('logoutBtn').addEventListener('click',logout);$('searchInput').addEventListener('input',()=>{page=1;renderRows();});['scoreFilter','statusFilter','websiteClassFilter','pipelineFilter'].forEach(id=>$(id).addEventListener('change',()=>{page=1;loadData();}));$('pageSize').addEventListener('change',e=>{pageSize=Number(e.target.value);page=1;loadData();});$('nextPageBtn').addEventListener('click',()=>{if(hasNextPage){page+=1;loadData();}});$('refreshBtn').addEventListener('click',loadData);$('jobsRefreshBtn').addEventListener('click',()=>loadJobs());$('startDemoJobBtn').addEventListener('click',startDemoJob);$('sourcesRefreshBtn').addEventListener('click',loadSources);$('sourceForm').addEventListener('submit',saveSource);$('sourceType').addEventListener('change',e=>{$('sourceCsvField').hidden=e.target.value!=='csv';});$('discoveryForm').addEventListener('submit',e=>{e.preventDefault();runDiscovery(true);});$('discoveryRunBtn').addEventListener('click',()=>runDiscovery(false));$('sourcesList').addEventListener('click',e=>{const button=e.target.closest?.('[data-source-action]');if(button)sourceAction(button.dataset.sourceId,button.dataset.sourceAction);});$('addForm').addEventListener('submit',addProspect);$('csvInput').addEventListener('change',e=>{const file=e.target.files[0];if(file){const reader=new FileReader();reader.onload=()=>renderCsv(parseCsv(reader.result));reader.readAsText(file);}});$('menuBtn').addEventListener('click',()=>document.querySelector('.sidebar').classList.toggle('open'));document.querySelectorAll('[data-scroll]').forEach(b=>b.addEventListener('click',()=>document.querySelector(b.dataset.scroll)?.scrollIntoView())); bootstrap(); })(); diff --git a/apps/web/smoke-test.html b/apps/web/smoke-test.html index 623b5e5..c01474f 100644 --- a/apps/web/smoke-test.html +++ b/apps/web/smoke-test.html @@ -33,5 +33,9 @@ frame.onload=async()=>{const d=frame.contentDocument;const js=await fetch('app.j ,['Deduplication review UI contract',()=>!!d.querySelector('#mergeDialog')&&js.includes('/matches')&&js.includes('review-required')&&js.includes('confidence')&&js.includes('reasons')] ,['Merge actions are explicit and reversible',()=>js.includes('/merge-history')&&js.includes('/reverse')&&js.includes('This action is reversible')&&js.includes('Confirm merge')&&!js.includes('autoMerge')] ,['Deduplication loading and errors',()=>js.includes('Loading match suggestions')&&js.includes('Unable to load match suggestions')&&js.includes('merge-history')&&js.includes('Unable to load merge history')] + ,['Domain intelligence smoke marker and controls',()=>!!d.querySelector('[data-smoke="domain-intelligence"]')&&js.includes('/domains/check')&&js.includes('/domain-candidates')&&js.includes('runDomainCheckBtn')] + ,['DNS capability and uncertainty labels',()=>['DNS checks','A','AAAA','MX','NS','TXT','NXDOMAIN','Unknown','Error','Cache'].every(x=>js.includes(x))] + ,['Candidate confidence, reasons, and safe availability',()=>js.includes('confidence')&&js.includes('reasons')&&js.includes('check-availability')&&js.includes('Not configured')&&js.includes('No ownership conclusion')&&!js.includes('Domain is available')] + ,['Domain loading, error, tenant, and auth states',()=>js.includes('Loading DNS results')&&js.includes('Unable to load DNS results')&&js.includes('Tenant/workspace access denied')&&js.includes("credentials:'include'")] ];let passed=0;document.querySelector('#checks').innerHTML=checks.map(([name,test])=>{const ok=test();if(ok)passed++;return `