add deterministic pilot benchmarks

2026-09-03 12:50:00 +02:00
parent 6b41d5b9ee
commit 9622f76977
11 changed files with 763 additions and 0 deletions
@@ -0,0 +1,19 @@
+# Phase 16 pilot benchmark
+
+Run from the repository root:
+
+```bash
+python3 scripts/benchmark_phase16.py
+```
+
+The command is offline-only and writes `docs/benchmarks/phase16.latest.json`. Use
+`--no-latency` when comparing deterministic result sections in CI. The committed
+fixture is `apps/api/fixtures/phase16.json`; it is intentionally synthetic and
+small. The JSON report repeats its limitations and acceptance thresholds and is
+validated structurally by `phase16.schema.json`.
+
+Acceptance thresholds are normalization accuracy 100%; fuzzy-match precision and
+recall at least 90%; contact precision and recall at least 85%; website fixture
+accuracy at least 90%; reproducible scoring; and zero tenant leakage. Timing and
+cache ratios are diagnostic pilot measurements, not production capacity claims.
+No benchmark performs network outreach or website fetching.