add deterministic pilot benchmarks

README.md

@@ -86,6 +86,18 @@ A discovery query is a tenant-scoped, bounded, auditable request that can be val
 
 SQLite, the in-process worker, and the named local volume are suitable for the pilot only; production migration, durable queue/worker leases, event retention/backup, SSE delivery, and tested backup/restore remain unfinished. Redis and Celery are not implemented. The development password fallback is PBKDF2 rather than production Argon2id. Before production, complete the gates in `docs/SECURITY.md` and `docs/OPERATIONS.md`, including MFA, TLS, CSRF protection, source approval and terms review, rate limiting, circuit monitoring, tenant-scoped job/event authorization, idempotent side-effect handling, durable raw-source/audit retention, SSRF-safe fetching if a future scanner is approved, and tested backups/restores.
 
+## Phase 16 pilot benchmark boundary
+
+Phase 16 defines a local, reproducible pilot methodology for evaluating classification and scoring against labeled **synthetic fixtures**. A fixture has a stable ID, schema/label-policy version, input, expected class and safety state, and rationale; it contains no live personal data, credentials, uncontrolled network response, or production record. The fixture manifest is kept separate from implementation outputs and is versioned/hash-recorded with every run. Positive, negative, unknown, blocked, partial, timeout, error, stale, conflicting, suppressed, malformed, boundary, cache, and cross-tenant cases must be represented where applicable.
+
+Reports include a confusion matrix and support plus precision (`TP/(TP+FP)`) and recall (`TP/(TP+FN)`) for each declared class, with `N/A` for zero denominators. Unknown/blocked/partial/error and safety outcomes are reported separately. Synthetic precision/recall is not evidence of identity, ownership, consent, deliverability, fairness, generalization, operational reliability, or production usefulness; labels may be cleaner or less representative than real data. See `docs/BENCHMARKS.md` for the complete protocol, limitations, reproducibility fields, human-review sampling, and acceptance thresholds.
+
+Classification/scoring reproducibility requires identical deterministic output fields across repeated fresh-process runs, using recorded commit, fixture hash, algorithm/rule-set versions, runtime/dependency/image versions, non-secret configuration, seed, locale/timezone, rounding, and tie-breaking. Latency must report cold and warm cache separately with cache state, freshness, hit/miss counts, sample count, median, p95, p99, maximum, errors, and timeout handling; averages alone are insufficient. A cache hit is never a fresh observation.
+
+The pilot gates are synthetic-only: deterministic fields must reproduce at 100%; where positive support is at least 20, precision must be `>=0.90` and recall `>=0.85`; suppressed/blocked/malformed/unknown safety cases must remain non-eligible/non-positive as applicable; tenant-isolation probes must pass 100%; and every latency sample must retain a duration and cache-state label. Human review is blinded and reproducible, stratified across predictions, uncertainty/suppression, and disagreements, with at least 20 fixtures or 10% of the set (whichever is larger, capped by the declared budget). Passing these gates makes no production claim.
+
+Remote push and deployment prerequisites remain unresolved. Local docs, tests, Compose validation, or a local benchmark do not constitute a remote push or production deployment. Remote publication still requires repository authentication and branch permission; deployment still requires the Phase 15 reviewed revision, host/Docker/Compose access, protected directory, secrets, DNS/TLS/Virtualmin, encrypted off-host backups, monitoring, rollback ownership, and operational approval. No deployment is performed by this change.
+
 ## Phase 6 normalization and deduplication boundary
 
 Normalization is deterministic and versioned. For South African data, phone values are stripped to digits, local 10-digit `0` forms and `00 27` forms are converted to canonical `+27...`, and unknown international numbers retain their explicit country code; presentation punctuation must not create a second identity. Locations derive whitespace/case/diacritic-folded province, city, and suburb fields. A normalized value is not proof that the underlying observation is correct.