add reversible prospect deduplication
This commit is contained in:
@@ -41,6 +41,16 @@ Before enabling any adapter, verify the registry entry has a stable ID/version,
|
||||
|
||||
Monitor per-source request counts, rate-limit responses, latency, errors, circuit state, and raw-record retention/deletion outcomes. On rate-limit, terms, approval, or circuit-open conditions, fail closed, preserve a safe audit event, and report deferred/unavailable rather than an empty result. Do not retry through another source or reset a circuit manually without an approved incident/change record. The current stack has no live source to monitor; these controls must precede any future implementation.
|
||||
|
||||
## Phase 6 normalization and deduplication operations
|
||||
|
||||
Normalization and duplicate review are data-integrity operations, not discovery. Record the normalization and algorithm versions with every derived SA phone/location value and suggestion. Verify that local South African phone forms are interpreted only with explicit `+27` context, that original values remain available, and that ambiguous locations are flagged rather than guessed. Re-running the same input/version must produce the same canonical values, score, band, and reasons.
|
||||
|
||||
Operate the review queue with the fixed bands: `>=0.90` strong suggestion, `0.75–0.8999` review suggestion, `<0.75` no suggestion. No band authorizes a merge. A permitted human must choose the survivor and explicitly confirm; operators must be able to reject a suggestion. Before any merge, verify the tenant, permissions, conflict list, and that a complete pre-merge snapshot was durably written. Preserve every child, evidence/provenance item, source-record/raw lineage link, note, pipeline state, and audit entry; never resolve a conflict by deleting the losing source record.
|
||||
|
||||
After a merge, read back the survivor and audit trail, check child/evidence/source-record counts and provenance links, and record the snapshot ID. For reversal, use only the snapshot, verify the same tenant and authorization, restore the prior graph transactionally, and read back the restored records and audit event. If snapshot creation, preservation, or audit append fails, stop the mutation and investigate; do not retry blindly. Cross-tenant candidates must be absent, not merely hidden by the UI.
|
||||
|
||||
The current Compose/MVP runtime exposes the Phase 6 match-suggestion, explicit merge-confirmation, tenant-scoped merge-history, and reversal routes; no separate worker is required. It has basic canonicalization and exact duplicate handling plus pilot-grade fuzzy review. Monitor merge audit events and verify child/source-record counts after each operation. Do not schedule or report deduplication as automated discovery, enrichment, verification, or outreach.
|
||||
|
||||
## Phase 4 jobs and live logging
|
||||
|
||||
The Phase 4 MVP provides SQLite-backed job status/detail/event routes and a browser monitor. A job moves `queued` → `running` → `succeeded`/`failed`/`cancelled`, retains its attempt and tenant identity, and appends per-job events with a monotonic sequence cursor. Operators inspect status and replay events by polling; SSE may provide lower-latency delivery but is not implemented and must replay from the persisted cursor and fall back to polling after disconnects.
|
||||
|
||||
Reference in New Issue
Block a user