MarketingTool/README.md

# Prospect Intelligence Platform

A safety-first Phase 4 design/implementation boundary for **manual**, evidence-led prospect qualification and the future job/live-log workflow. The current runtime remains the Phase 3 manual vertical slice: it stores tenant-owned businesses and child intelligence records, keeps provenance with each evidence item, supports a review pipeline, and records operationally relevant changes. **Automated discovery, DNS/website scanning, and outreach are not part of this release. Automated outreach is disabled.**

## Included

- Dependency-free Python/SQLite API under `apps/api`.
- Tenant-scoped business detail APIs with child intelligence/evidence records, provenance fields, notes, pipeline state, and audit history.
- Server-side normalization, conservative website classification, exact deduplication, versioned scoring, and suppression checks.
- Bounded list pagination and server-side filters so a tenant cannot request an unbounded prospect collection.
- Responsive static dashboard under `apps/web` with authenticated explorer filters, paginated results, detail review, manual intake, notes/pipeline context, evidence provenance, and browser-only CSV preview.
- Docker Compose runtime with non-root containers, read-only filesystems, health checks, and a named SQLite data volume.
- Browser authentication with server-side sessions and an optional first-run admin bootstrap.
- Phase 4 MVP job monitor and SQLite-backed job/event schema/API surface, with the production limitations documented below.

## Current workflow and Phase 4 boundary

1. A permitted workspace member manually creates or reviews a prospect.
2. The business detail response is the aggregate record for that tenant; related intelligence/evidence rows are returned only through the tenant-scoped detail surface.
3. Each manually entered intelligence item should retain its source/provenance (for example, source label or URL, observed value, and captured/verified time). Missing provenance is a data-quality limitation, not permission to infer facts.
4. Members use the pipeline state and notes to coordinate human review. A state change or note is an application event and is included in the record's audit/activity history where exposed by the API.
5. Suppression remains a hard safety boundary. Suppressed or unreviewed records must not be treated as eligible for contact.

The API applies the organization/tenant boundary server-side to list, detail, child-record, notes, pipeline, and audit reads and writes. Clients must use the returned pagination metadata and follow `next`/`previous` links or tokens rather than assuming that one response contains the whole tenant dataset. See `apps/api/README.md` for the route contract and limits.

### Phase 4 jobs/live logging contract

The planned asynchronous contract is: create one tenant-scoped job, return a stable job identifier, and move it through `queued` → `running` → a terminal state (`succeeded`, `failed`, `cancelled`). Each accepted request should carry an idempotency key whose scope and request fingerprint prevent duplicate jobs while allowing a safe replay of the original result. A job should persist append-only events with a monotonically increasing per-job sequence number, timestamp, level/type, safe message, and job/tenant identifiers.

Clients should poll a tenant-scoped job status/events endpoint using `after_sequence` (or an equivalent cursor), with bounded backoff and terminal-state handling. SSE is a planned low-latency delivery option, not a current implementation; polling remains the compatibility fallback. Cancellation and retry must be explicit, authorized controls: cancellation is cooperative and may finish as `cancelled` or report that the job is already terminal; retry creates a new attempt while retaining the original job/idempotency lineage and must not duplicate side effects.

The current MVP has SQLite job/event persistence, job status/list/detail and event APIs, cancellation/retry controls, and a browser job monitor that polls while work is active. SSE is not implemented; it remains a future delivery optimization over the persisted cursor. There is no Redis/Celery worker: the current in-process worker is suitable only for development/pilot use and must not be treated as durable, horizontally scalable execution.

## Run locally

```bash
cd apps/api
python3 -m unittest discover -v
python3 app/main.py --host 127.0.0.1 --port 8000 --db /tmp/prospects.db
```

Serve the UI separately:

```bash
cd apps/web
python3 -m http.server 8080
```

Open `http://127.0.0.1:8080`. Set `window.API_BASE` in the browser console to `http://127.0.0.1:8000` when testing the authenticated API locally, then sign in with the configured workspace credentials.

## API smoke calls

```bash
curl http://127.0.0.1:8000/api/v1/health/live
curl 'http://127.0.0.1:8000/api/v1/businesses?page=1&page_size=25&pipeline_stage=new'
curl http://127.0.0.1:8000/api/v1/businesses/1
curl -X POST http://127.0.0.1:8000/api/v1/businesses \
  -H 'content-type: application/json' \
  -d '{"name":"Example Plumbing","website":"https://example.invalid","email":"info@example.invalid","phone":"+27 21 555 0100"}'
```

The protected calls require the authenticated session cookie. Exact child-record, notes, pipeline, and audit routes are documented in `apps/api/README.md` and are never cross-tenant addressable by changing an ID.

## Compose

```bash
cp .env.example .env
docker compose config --quiet
docker compose up --build -d
curl -fsS http://localhost:8000/api/v1/health/live
curl -fsS http://localhost:8080/healthz
docker compose down
```

Compose passes the optional `BOOTSTRAP_ADMIN_EMAIL` and `BOOTSTRAP_ADMIN_PASSWORD` values to the API. Set both in an untracked `.env` only when provisioning a fresh instance, then remove them and rotate the password after the bootstrap admin is created. No credentials belong in this repository.

Authenticated browser requests use a server-side session cookie; login creates a session and logout invalidates it. The liveness endpoints (`GET /api/v1/health/live` and `GET /healthz`) intentionally remain unauthenticated so Docker, ingress, and monitoring health checks can use them. Authentication is not a substitute for tenant/authorization checks: protected routes must enforce the session and organization boundary server-side.

## Explicit non-goals and remaining limitations

This Phase 4 boundary still has no automated discovery, DNS resolution, website/HTTP scanning, enrichment scheduler, external source adapter, email/SMS sender, or outreach endpoint. CSV remains a browser/API preview flow and does not silently persist rows. SQLite, the in-process worker, and the named local volume are suitable for the pilot only; production migration, durable queue/worker leases, event retention/backup, SSE delivery, and tested backup/restore remain unfinished. Redis and Celery are not implemented. The development password fallback is PBKDF2 rather than production Argon2id. Before production, complete the gates in `docs/SECURITY.md` and `docs/OPERATIONS.md`, including MFA, TLS, CSRF protection, rate limiting, tenant-scoped job/event authorization, idempotent side-effect handling, durable audit/event retention, migrations, approved source policy, SSRF-safe fetching if a future scanner is approved, and tested backups/restores.

## Verification

```bash
python3 -m unittest discover -v -s apps/api/tests -t apps/api
python3 -m compileall -q apps/api apps/web
git diff --check
docker compose config --quiet
```

See `apps/api/README.md`, `apps/web/README.md`, `docs/SECURITY.md`, and `docs/OPERATIONS.md` for details.