root/MarketingTool
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

add reversible prospect deduplication

Browse Source
This commit is contained in:
Marco0300
2026-09-03 08:46:22 +02:00
parent 46cc1f6182
commit 25ee7931ab
14 changed files with 290 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+12 -2
View File
@@ -1,12 +1,12 @@
# Prospect Intelligence Platform
A safety-first Phase 5 design/implementation boundary for **manual**, evidence-led prospect qualification and controlled source ingestion. The current runtime remains a manual vertical slice: it stores tenant-owned businesses and child intelligence records, keeps provenance with each evidence item, supports a review pipeline, and records operationally relevant changes. Phase 5 defines source adapters, discovery-query records, raw-source retention, and health controls; it does **not** enable network discovery. **Automated outreach is disabled, and no live source may be enabled without explicit approval.**
A safety-first Phase 6 design/implementation boundary for **manual**, evidence-led prospect qualification and controlled source ingestion. The current runtime remains a manual vertical slice: it stores tenant-owned businesses and child intelligence records, keeps provenance with each evidence item, supports a review pipeline, and records operationally relevant changes. Phase 6 defines deterministic South African normalization and deduplication review semantics in addition to the Phase 5 source controls; it does **not** enable network discovery. **Automated outreach is disabled, and no live source may be enabled without explicit approval.**
## Included
- Dependency-free Python/SQLite API under `apps/api`.
- Tenant-scoped business detail APIs with child intelligence/evidence records, provenance fields, notes, pipeline state, and audit history.
- Server-side normalization, conservative website classification, exact deduplication, versioned scoring, and suppression checks.
- Server-side normalization, conservative website classification, exact deduplication, versioned scoring, and suppression checks. Phase 6 documents the SA phone/location canonical forms and the review-only fuzzy-match contract.
- Bounded list pagination and server-side filters so a tenant cannot request an unbounded prospect collection.
- Responsive static dashboard under `apps/web` with authenticated explorer filters, paginated results, detail review, manual intake, notes/pipeline context, evidence provenance, and browser-only CSV preview.
- Docker Compose runtime with non-root containers, read-only filesystems, health checks, and a named SQLite data volume.
@@ -86,6 +86,16 @@ A discovery query is a tenant-scoped, bounded, auditable request that can be val
SQLite, the in-process worker, and the named local volume are suitable for the pilot only; production migration, durable queue/worker leases, event retention/backup, SSE delivery, and tested backup/restore remain unfinished. Redis and Celery are not implemented. The development password fallback is PBKDF2 rather than production Argon2id. Before production, complete the gates in `docs/SECURITY.md` and `docs/OPERATIONS.md`, including MFA, TLS, CSRF protection, source approval and terms review, rate limiting, circuit monitoring, tenant-scoped job/event authorization, idempotent side-effect handling, durable raw-source/audit retention, SSRF-safe fetching if a future scanner is approved, and tested backups/restores.
## Phase 6 normalization and deduplication boundary
Normalization is deterministic and versioned. For South African data, phone values are stripped to digits, local 10-digit `0` forms and `00 27` forms are converted to canonical `+27...`, and unknown international numbers retain their explicit country code; presentation punctuation must not create a second identity. Locations derive whitespace/case/diacritic-folded province, city, and suburb fields. A normalized value is not proof that the underlying observation is correct.
Exact keys (for example, canonical domain, email, or phone) may identify duplicate candidates. Fuzzy matching is deterministic and suggestion-only: the same inputs and normalization version produce the same candidate, score, and reason. A suggested match must never merge automatically. Use the documented thresholds: `>=0.90` is a strong suggestion, `0.75–0.8999` is a review suggestion, and `<0.75` is not surfaced as a suggestion. A human with permission must explicitly confirm each merge.
Every confirmed merge must create a tenant-scoped, immutable-enough merge snapshot before mutation, recording the surviving and absorbed IDs, normalized comparison inputs, score/reasons, acting user, timestamp, and schema/normalization versions. The operation must be reversible from that snapshot. It must preserve or re-parent every child, evidence item, provenance/source-record link, note, pipeline/audit history, and original source identity; conflicts remain visible for human resolution rather than being silently overwritten. Cross-tenant candidates are never comparable or mergeable, and each suggestion, confirmation, rejection, reversal, and preservation/conflict decision belongs in the audit trail.
The MVP now exposes deterministic match suggestions at `GET /api/v1/businesses/{id}/matches`, explicit merge confirmation in the web review dialog, tenant-scoped merge history, and `POST /api/v1/merge-history/{id}/reverse`. The implementation remains a pilot boundary: hardening is still needed for a dedicated merge permission, stronger server-side confirmation semantics, full snapshot conflict handling, and production-grade rollback guarantees. Do not describe a normalized or suggested match as verified identity, discovery, enrichment, or outreach authorization.
## Verification
```bash