# Prospect Intelligence Platform

A safety-first Phase 7 design/implementation boundary for **manual**, evidence-led prospect qualification, controlled source ingestion, and domain intelligence review. The current runtime remains a manual vertical slice: it stores tenant-owned businesses and child intelligence records, keeps provenance with each evidence item, supports a review pipeline, and records operationally relevant changes. Phase 7 defines conservative registrable-domain/PSL and DNS observation semantics in addition to the Phase 6 normalization and deduplication rules; it does **not** enable network discovery. **Automated outreach is disabled, and no live source may be enabled without explicit approval.**

## Included

- Dependency-free Python/SQLite API under `apps/api`.
- Tenant-scoped business detail APIs with child intelligence/evidence records, provenance fields, notes, pipeline state, and audit history.
- Server-side normalization, conservative website classification, exact deduplication, versioned scoring, and suppression checks. Phase 6 documents the SA phone/location canonical forms and the review-only fuzzy-match contract.
- Bounded list pagination and server-side filters so a tenant cannot request an unbounded prospect collection.
- Responsive static dashboard under `apps/web` with authenticated explorer filters, paginated results, detail review, manual intake, notes/pipeline context, evidence provenance, and browser-only CSV preview.
- Docker Compose runtime with non-root containers, read-only filesystems, health checks, and a named SQLite data volume.
- Browser authentication with server-side sessions and an optional first-run admin bootstrap.
- Phase 4 MVP job monitor and SQLite-backed job/event schema/API surface, with the production limitations documented below.
- Phase 5 source-ingestion contract: an approved source registry owns adapter terms, rate limits, retention, and health/circuit policy; CSV and manual reference adapters are the safe initial adapters.
- Discovery queries are recorded as bounded, auditable intent and dry-run plans. Recording a query does not perform network discovery or imply that results exist.

## Current workflow and Phase 4/5 boundary

1. A permitted workspace member manually creates or reviews a prospect.
2. The business detail response is the aggregate record for that tenant; related intelligence/evidence rows are returned only through the tenant-scoped detail surface.
3. Each manually entered intelligence item should retain its source/provenance (for example, source label or URL, observed value, and captured/verified time). Missing provenance is a data-quality limitation, not permission to infer facts.
4. Members use the pipeline state and notes to coordinate human review. A state change or note is an application event and is included in the record's audit/activity history where exposed by the API.
5. Suppression remains a hard safety boundary. Suppressed or unreviewed records must not be treated as eligible for contact.

The API applies the organization/tenant boundary server-side to list, detail, child-record, notes, pipeline, and audit reads and writes. Clients must use the returned pagination metadata and follow `next`/`previous` links or tokens rather than assuming that one response contains the whole tenant dataset. See `apps/api/README.md` for the route contract and limits.

### Phase 4 jobs/live logging contract

The planned asynchronous contract is: create one tenant-scoped job, return a stable job identifier, and move it through `queued` → `running` → a terminal state (`succeeded`, `failed`, `cancelled`). Each accepted request should carry an idempotency key whose scope and request fingerprint prevent duplicate jobs while allowing a safe replay of the original result. A job should persist append-only events with a monotonically increasing per-job sequence number, timestamp, level/type, safe message, and job/tenant identifiers.

Clients should poll a tenant-scoped job status/events endpoint using `after_sequence` (or an equivalent cursor), with bounded backoff and terminal-state handling. SSE is a planned low-latency delivery option, not a current implementation; polling remains the compatibility fallback. Cancellation and retry must be explicit, authorized controls: cancellation is cooperative and may finish as `cancelled` or report that the job is already terminal; retry creates a new attempt while retaining the original job/idempotency lineage and must not duplicate side effects.

The current MVP has SQLite job/event persistence, job status/list/detail and event APIs, cancellation/retry controls, and a browser job monitor that polls while work is active. SSE is not implemented; it remains a future delivery optimization over the persisted cursor. There is no Redis/Celery worker: the current in-process worker is suitable only for development/pilot use and must not be treated as durable, horizontally scalable execution.

## Run locally

```bash
cd apps/api
python3 -m unittest discover -v
python3 app/main.py --host 127.0.0.1 --port 8000 --db /tmp/prospects.db
```

Serve the UI separately:

```bash
cd apps/web
python3 -m http.server 8080
```

Open `http://127.0.0.1:8080`. Set `window.API_BASE` in the browser console to `http://127.0.0.1:8000` when testing the authenticated API locally, then sign in with the configured workspace credentials.

## API smoke calls

```bash
curl http://127.0.0.1:8000/api/v1/health/live
curl 'http://127.0.0.1:8000/api/v1/businesses?page=1&page_size=25&pipeline_stage=new'
curl http://127.0.0.1:8000/api/v1/businesses/1
curl -X POST http://127.0.0.1:8000/api/v1/businesses \
  -H 'content-type: application/json' \
  -d '{"name":"Example Plumbing","website":"https://example.invalid","email":"info@example.invalid","phone":"+27 21 555 0100"}'
```

The protected calls require the authenticated session cookie. Exact child-record, notes, pipeline, and audit routes are documented in `apps/api/README.md` and are never cross-tenant addressable by changing an ID.

## Compose

```bash
cp .env.example .env
docker compose config --quiet
docker compose up --build -d
curl -fsS http://localhost:8000/api/v1/health/live
curl -fsS http://localhost:8080/healthz
docker compose down
```

Compose passes the optional `BOOTSTRAP_ADMIN_EMAIL` and `BOOTSTRAP_ADMIN_PASSWORD` values to the API. Set both in an untracked `.env` only when provisioning a fresh instance, then remove them and rotate the password after the bootstrap admin is created. No credentials belong in this repository.

Authenticated browser requests use a server-side session cookie; login creates a session and logout invalidates it. The liveness endpoints (`GET /api/v1/health/live` and `GET /healthz`) intentionally remain unauthenticated so Docker, ingress, and monitoring health checks can use them. Authentication is not a substitute for tenant/authorization checks: protected routes must enforce the session and organization boundary server-side.

## Phase 5 source boundary and remaining limitations

Phase 5 defines a source adapter contract and registry; it does not implement network discovery, DNS resolution, website/HTTP scanning, enrichment scheduling, or a live external-source adapter. A source adapter must declare its identity, terms owner, permitted purpose, rate limits, retention class, query/result schema, dry-run behavior, and health/circuit controls. CSV and manual reference adapters may be used for operator-supplied data; they must preserve source attribution and raw source records, and must not silently turn preview data into outreach or verified facts.

A discovery query is a tenant-scoped, bounded, auditable request that can be validated and dry-run without contacting a source. Any live source requires explicit product/legal/security approval, a registered adapter, and an operational enablement decision; absent all three, execution must fail closed. Circuit-open, rate-limit, terms, or approval failures must produce a safe non-live result. Raw source records are retained only under the approved retention class and must exclude secrets and unnecessary personal data.

SQLite, the in-process worker, and the named local volume are suitable for the pilot only; production migration, durable queue/worker leases, event retention/backup, SSE delivery, and tested backup/restore remain unfinished. Redis and Celery are not implemented. The development password fallback is PBKDF2 rather than production Argon2id. Before production, complete the gates in `docs/SECURITY.md` and `docs/OPERATIONS.md`, including MFA, TLS, CSRF protection, source approval and terms review, rate limiting, circuit monitoring, tenant-scoped job/event authorization, idempotent side-effect handling, durable raw-source/audit retention, SSRF-safe fetching if a future scanner is approved, and tested backups/restores.

## Phase 6 normalization and deduplication boundary

Normalization is deterministic and versioned. For South African data, phone values are stripped to digits, local 10-digit `0` forms and `00 27` forms are converted to canonical `+27...`, and unknown international numbers retain their explicit country code; presentation punctuation must not create a second identity. Locations derive whitespace/case/diacritic-folded province, city, and suburb fields. A normalized value is not proof that the underlying observation is correct.

Exact keys (for example, canonical domain, email, or phone) may identify duplicate candidates. Fuzzy matching is deterministic and suggestion-only: the same inputs and normalization version produce the same candidate, score, and reason. A suggested match must never merge automatically. Use the documented thresholds: `>=0.90` is a strong suggestion, `0.75–0.8999` is a review suggestion, and `<0.75` is not surfaced as a suggestion. A human with permission must explicitly confirm each merge.

Every confirmed merge must create a tenant-scoped, immutable-enough merge snapshot before mutation, recording the surviving and absorbed IDs, normalized comparison inputs, score/reasons, acting user, timestamp, and schema/normalization versions. The operation must be reversible from that snapshot. It must preserve or re-parent every child, evidence item, provenance/source-record link, note, pipeline/audit history, and original source identity; conflicts remain visible for human resolution rather than being silently overwritten. Cross-tenant candidates are never comparable or mergeable, and each suggestion, confirmation, rejection, reversal, and preservation/conflict decision belongs in the audit trail.

The MVP now exposes deterministic match suggestions at `GET /api/v1/businesses/{id}/matches`, explicit merge confirmation in the web review dialog, tenant-scoped merge history, and `POST /api/v1/merge-history/{id}/reverse`. The implementation remains a pilot boundary: hardening is still needed for a dedicated merge permission, stronger server-side confirmation semantics, full snapshot conflict handling, and production-grade rollback guarantees. Do not describe a normalized or suggested match as verified identity, discovery, enrichment, or outreach authorization.

## Phase 7 domain intelligence boundary

Domain intelligence is an observation and review aid, not proof of business identity, control of a domain, or availability. A domain normalizer may derive a lowercase ASCII/Unicode comparison form and a **registrable domain** using a versioned Public Suffix List (PSL). The PSL is an input with update/version drift: unknown, private, malformed, single-label, localhost, and IP-literal values must remain unresolved rather than guessed. A subdomain is not automatically a separate candidate, and a public suffix itself is never a registrable domain.

DNS status is explicit: `not_checked`, `pending`, `resolved`, `nxdomain`, `no_data`, `timeout`, `servfail`, `blocked`, and `error` describe the check outcome, not a business conclusion. MX, NS, and TXT observations may be absent, partial, truncated, stale, resolver-dependent, or blocked; no record is not proof that mail, delegation, ownership, or a business relationship is absent. Store the resolver/source, observed time, TTL where supplied, and uncertainty/error metadata. Caches must be bounded and keyed by normalized query/type/class plus resolver policy, honor an observed TTL without extending authority, and expose freshness/staleness; cached data must never be presented as a fresh check.

Association confidence is separate from DNS status and from duplicate score. It must be derived from explainable, tenant-scoped evidence (for example, an operator citation, an exact business-domain observation, or corroborating DNS facts), retain the algorithm/version and uncertainty reasons, and remain suggestion-only. Candidate generation must reject cross-tenant records, public-suffix-only values, malformed or IP-only inputs, and suppressed/merged targets as applicable; it must not auto-attach a domain or infer ownership from a shared, parked, wildcard, sibling-subdomain, homograph, or merely resolvable domain. Every candidate needs human review, provenance, and an auditable accept/reject decision.

The platform must not claim that a domain is available, unregistered, or safe to acquire without an explicitly authorized availability provider registered with current product/legal/security approval, terms, tenant scope, rate limits, retention, and operational enablement. DNS `nxdomain` or `no_data` is not an availability result. Provider outages, rate limits, stale responses, conflicting results, and unknown status must remain `unknown`/`unavailable`, fail closed, and never trigger purchase, outreach, or automated follow-up.

Phase 7 remains a documentation/contract boundary in this MVP: there is no live DNS resolver, PSL-backed enrichment worker, cache service, or availability provider in Compose. Production work still includes selecting and versioning the PSL, implementing bounded DNS resolution and TTL-aware cache invalidation, defining MX/NS/TXT parsing and uncertainty retention, adding association review/permission/audit tests, and completing an approved availability-provider integration with SSRF/network egress controls, monitoring, retention, and incident/rollback procedures.

## Verification

```bash
python3 -m unittest discover -v -s apps/api/tests -t apps/api
python3 -m compileall -q apps/api apps/web
git diff --check
docker compose config --quiet
```

See `apps/api/README.md`, `apps/web/README.md`, `docs/SECURITY.md`, and `docs/OPERATIONS.md` for details.