MarketingTool/README.md

# Prospect Intelligence Platform

A safety-first Phase 5 design/implementation boundary for **manual**, evidence-led prospect qualification and controlled source ingestion. The current runtime remains a manual vertical slice: it stores tenant-owned businesses and child intelligence records, keeps provenance with each evidence item, supports a review pipeline, and records operationally relevant changes. Phase 5 defines source adapters, discovery-query records, raw-source retention, and health controls; it does **not** enable network discovery. **Automated outreach is disabled, and no live source may be enabled without explicit approval.**

## Included

- Dependency-free Python/SQLite API under `apps/api`.
- Tenant-scoped business detail APIs with child intelligence/evidence records, provenance fields, notes, pipeline state, and audit history.
- Server-side normalization, conservative website classification, exact deduplication, versioned scoring, and suppression checks.
- Bounded list pagination and server-side filters so a tenant cannot request an unbounded prospect collection.
- Responsive static dashboard under `apps/web` with authenticated explorer filters, paginated results, detail review, manual intake, notes/pipeline context, evidence provenance, and browser-only CSV preview.
- Docker Compose runtime with non-root containers, read-only filesystems, health checks, and a named SQLite data volume.
- Browser authentication with server-side sessions and an optional first-run admin bootstrap.
- Phase 4 MVP job monitor and SQLite-backed job/event schema/API surface, with the production limitations documented below.
- Phase 5 source-ingestion contract: an approved source registry owns adapter terms, rate limits, retention, and health/circuit policy; CSV and manual reference adapters are the safe initial adapters.
- Discovery queries are recorded as bounded, auditable intent and dry-run plans. Recording a query does not perform network discovery or imply that results exist.

## Current workflow and Phase 4/5 boundary

1. A permitted workspace member manually creates or reviews a prospect.
2. The business detail response is the aggregate record for that tenant; related intelligence/evidence rows are returned only through the tenant-scoped detail surface.
3. Each manually entered intelligence item should retain its source/provenance (for example, source label or URL, observed value, and captured/verified time). Missing provenance is a data-quality limitation, not permission to infer facts.
4. Members use the pipeline state and notes to coordinate human review. A state change or note is an application event and is included in the record's audit/activity history where exposed by the API.
5. Suppression remains a hard safety boundary. Suppressed or unreviewed records must not be treated as eligible for contact.

The API applies the organization/tenant boundary server-side to list, detail, child-record, notes, pipeline, and audit reads and writes. Clients must use the returned pagination metadata and follow `next`/`previous` links or tokens rather than assuming that one response contains the whole tenant dataset. See `apps/api/README.md` for the route contract and limits.

### Phase 4 jobs/live logging contract

The planned asynchronous contract is: create one tenant-scoped job, return a stable job identifier, and move it through `queued` → `running` → a terminal state (`succeeded`, `failed`, `cancelled`). Each accepted request should carry an idempotency key whose scope and request fingerprint prevent duplicate jobs while allowing a safe replay of the original result. A job should persist append-only events with a monotonically increasing per-job sequence number, timestamp, level/type, safe message, and job/tenant identifiers.

Clients should poll a tenant-scoped job status/events endpoint using `after_sequence` (or an equivalent cursor), with bounded backoff and terminal-state handling. SSE is a planned low-latency delivery option, not a current implementation; polling remains the compatibility fallback. Cancellation and retry must be explicit, authorized controls: cancellation is cooperative and may finish as `cancelled` or report that the job is already terminal; retry creates a new attempt while retaining the original job/idempotency lineage and must not duplicate side effects.

The current MVP has SQLite job/event persistence, job status/list/detail and event APIs, cancellation/retry controls, and a browser job monitor that polls while work is active. SSE is not implemented; it remains a future delivery optimization over the persisted cursor. There is no Redis/Celery worker: the current in-process worker is suitable only for development/pilot use and must not be treated as durable, horizontally scalable execution.

## Run locally

```bash
cd apps/api
python3 -m unittest discover -v
python3 app/main.py --host 127.0.0.1 --port 8000 --db /tmp/prospects.db
```

Serve the UI separately:

```bash
cd apps/web
python3 -m http.server 8080
```

Open `http://127.0.0.1:8080`. Set `window.API_BASE` in the browser console to `http://127.0.0.1:8000` when testing the authenticated API locally, then sign in with the configured workspace credentials.

## API smoke calls

```bash
curl http://127.0.0.1:8000/api/v1/health/live
curl 'http://127.0.0.1:8000/api/v1/businesses?page=1&page_size=25&pipeline_stage=new'
curl http://127.0.0.1:8000/api/v1/businesses/1
curl -X POST http://127.0.0.1:8000/api/v1/businesses \
  -H 'content-type: application/json' \
  -d '{"name":"Example Plumbing","website":"https://example.invalid","email":"info@example.invalid","phone":"+27 21 555 0100"}'
```

The protected calls require the authenticated session cookie. Exact child-record, notes, pipeline, and audit routes are documented in `apps/api/README.md` and are never cross-tenant addressable by changing an ID.

## Compose

```bash
cp .env.example .env
docker compose config --quiet
docker compose up --build -d
curl -fsS http://localhost:8000/api/v1/health/live
curl -fsS http://localhost:8080/healthz
docker compose down
```

Compose passes the optional `BOOTSTRAP_ADMIN_EMAIL` and `BOOTSTRAP_ADMIN_PASSWORD` values to the API. Set both in an untracked `.env` only when provisioning a fresh instance, then remove them and rotate the password after the bootstrap admin is created. No credentials belong in this repository.

Authenticated browser requests use a server-side session cookie; login creates a session and logout invalidates it. The liveness endpoints (`GET /api/v1/health/live` and `GET /healthz`) intentionally remain unauthenticated so Docker, ingress, and monitoring health checks can use them. Authentication is not a substitute for tenant/authorization checks: protected routes must enforce the session and organization boundary server-side.

## Phase 5 source boundary and remaining limitations

Phase 5 defines a source adapter contract and registry; it does not implement network discovery, DNS resolution, website/HTTP scanning, enrichment scheduling, or a live external-source adapter. A source adapter must declare its identity, terms owner, permitted purpose, rate limits, retention class, query/result schema, dry-run behavior, and health/circuit controls. CSV and manual reference adapters may be used for operator-supplied data; they must preserve source attribution and raw source records, and must not silently turn preview data into outreach or verified facts.

A discovery query is a tenant-scoped, bounded, auditable request that can be validated and dry-run without contacting a source. Any live source requires explicit product/legal/security approval, a registered adapter, and an operational enablement decision; absent all three, execution must fail closed. Circuit-open, rate-limit, terms, or approval failures must produce a safe non-live result. Raw source records are retained only under the approved retention class and must exclude secrets and unnecessary personal data.

SQLite, the in-process worker, and the named local volume are suitable for the pilot only; production migration, durable queue/worker leases, event retention/backup, SSE delivery, and tested backup/restore remain unfinished. Redis and Celery are not implemented. The development password fallback is PBKDF2 rather than production Argon2id. Before production, complete the gates in `docs/SECURITY.md` and `docs/OPERATIONS.md`, including MFA, TLS, CSRF protection, source approval and terms review, rate limiting, circuit monitoring, tenant-scoped job/event authorization, idempotent side-effect handling, durable raw-source/audit retention, SSRF-safe fetching if a future scanner is approved, and tested backups/restores.

## Verification

```bash
python3 -m unittest discover -v -s apps/api/tests -t apps/api
python3 -m compileall -q apps/api apps/web
git diff --check
docker compose config --quiet
```

See `apps/api/README.md`, `apps/web/README.md`, `docs/SECURITY.md`, and `docs/OPERATIONS.md` for details.