add gated outreach preparation
This commit is contained in:
Marco0300
@@ -176,6 +176,20 @@ Suppression endpoints accept only approved normalized identifier kinds and recor
|
||||
|
||||
Every CRM mutation and report/export operation emits an audit record with tenant, actor, action, target, before/after or bounded result, policy/version, timestamps, correlation/idempotency ID, and safe reason. Audit and retention reads use the same organization predicate. CRM records, contact references, suppression decisions, report snapshots, and audit details require explicit retention classes, deletion/legal-hold semantics, and redacted logs. These are the Phase 12 contract; production readiness additionally requires durable migrations, worker/retry behavior, report reproducibility, export authorization, retention jobs, and transition/outcome/suppression/cross-tenant tests.
|
||||
|
||||
## Phase 14 draft-only outreach API contract
|
||||
|
||||
Phase 14 is a preparation contract, not a delivery feature. A future authenticated route may create a tenant-scoped outreach **draft** from a bounded approved evidence set, but the current API exposes no send, delivery, campaign, SMTP, validation-message, or autonomous follow-up endpoint. The server must reject any request that attempts to send or that treats draft creation/approval as delivery. `AUTOMATED_OUTREACH_ENABLED=false` is the no-send default in Compose and must be enforced server-side, not only by the UI.
|
||||
|
||||
Draft creation must evaluate and persist gate results for tenant/recipient scope, normalized suppression/do-not-contact, consent or other documented legal basis, jurisdiction/channel policy, evidence permission/freshness, current eligibility, provider approval, and content policy. Suppression is an unconditional deny. Public contact data, pipeline state, score, verification metadata, or AI evidence citations do not establish consent, lawful basis, deliverability, or permission to contact. A blocked, missing, stale, conflicting, or uncertain gate returns an explicit non-send reason rather than an empty success.
|
||||
|
||||
Provider configuration is server-side and deny-by-default. A registered provider must include an allowlisted ID/version, purpose/capability, permitted tenant/data class, processing region and retention terms, timeout/payload limits, per-tenant and global rate caps, daily message/cost ceilings, health/circuit state, approval owner/expiry, and explicit operations enablement. Credentials are secret-manager references only and must never be accepted from clients or returned in responses/logs. Fallbacks, if ever enabled, must be pre-approved for the same purpose, data class, policy, caps, citations, and authority; provider failure, timeout, quota, circuit-open, or expired approval fails closed.
|
||||
|
||||
Every draft must retain recipient/channel, bounded redacted content or a safe content hash, evidence IDs/source citations, exact evidence snapshot hash and observed times, policy/provider versions, gate outcomes, actor, approval status/expiry, and correlation ID. Drafts are immutable or versioned: an edit creates a new version and invalidates approval. Approval/rejection is an explicit authorized human operation bound to the unchanged draft/evidence/policy hash; it must re-check tenant scope, suppression, legal/consent state, freshness, and provider approval, and record actor, time, reason, before/after status, and audit event. An approved draft still requires a separately authorized future send operation.
|
||||
|
||||
Any future send or other side effect must require an `Idempotency-Key` scoped to tenant, operation, draft version, recipient, provider, and policy fingerprint. Exact retries return the original result; a different request under the same key is rejected. Enforce rate/message/cost caps before attempts and across retries, fallbacks, and workers; use bounded retry/backoff and circuit breaking. Audit draft/gate/approval/provider/cap/suppression events with safe per-item outcomes and redacted payloads. No route may infer completion from request acceptance.
|
||||
|
||||
The Phase 14 implementation remains documentation-only/pilot preparation: there is no draft persistence/API, consent ledger, legal-policy evaluator, configured provider, durable approval queue, delivery adapter, bounce/complaint feedback, secret manager, or production-grade audit/retention workflow in the current runtime. Production requires those components plus DPA/provider and jurisdictional legal review, suppression synchronization, kill switch, rollback/revocation, deletion/legal-hold verification, and integration tests proving no-send default, citation/hash binding, stale/uncertain gate failure, cap enforcement, idempotent replay/conflict rejection, approval expiry, and cross-tenant isolation.
|
||||
|
||||
## Remaining limitations and production migration work
|
||||
|
||||
SQLite is a pilot store with an MVP job/event schema but no production migration runner, durable queue, scheduler, worker lease/recovery, durable backup command, or tested restore workflow. Authentication currently uses a development password fallback and does not by itself provide production Argon2id, MFA, CSRF protection, rate limiting, or a complete retention-grade audit system. Website scanning remains subject to pilot limits: no production egress proxy/isolation, distributed crawl coordinator, hardened resolver, or compliance-grade scan-history retention. Production work must add SSRF/DNS-rebinding/redirect-chain tests, egress policy, authenticated history/cache isolation, budget/abuse enforcement, durable result retention/deletion, observability, reviewed content/robots/terms policy, and the Phase 7 PSL/DNS/availability controls. Redis/Celery/Postgres remain future options, not implemented dependencies.
|
||||
|
||||
@@ -307,6 +307,167 @@ class ApiHandler(BaseHTTPRequestHandler):
|
||||
if suggestions is not None: item["suggestions"] = suggestions
|
||||
return item
|
||||
|
||||
# Phase 14: outreach is a reviewable preparation workflow only. There is
|
||||
# deliberately no provider client in this service and send never performs
|
||||
# network I/O.
|
||||
OUTREACH_PROVIDERS = {"smtp", "sendgrid", "twilio", "whatsapp"}
|
||||
OUTREACH_KINDS = {"email", "phone", "whatsapp"}
|
||||
OUTREACH_DAILY_CAP = 100
|
||||
OUTREACH_BATCH_CAP = 25
|
||||
|
||||
def _provider_json(self, row, org):
|
||||
if not row:
|
||||
return {"organization_id": org, "provider": "", "enabled": False,
|
||||
"policy": {"consent_required": True}, "daily_cap": self.OUTREACH_DAILY_CAP,
|
||||
"batch_cap": self.OUTREACH_BATCH_CAP}
|
||||
item = {"id": row["id"], "organization_id": org, "provider": row["provider"],
|
||||
"enabled": bool(row["enabled"]), "daily_cap": row["daily_cap"],
|
||||
"batch_cap": row["batch_cap"]}
|
||||
try: item["policy"] = json.loads(row["policy_json"] or "{}")
|
||||
except (TypeError, ValueError): item["policy"] = {"consent_required": True}
|
||||
return item
|
||||
|
||||
def provider_config(self, db, user, payload=None):
|
||||
org = user["organization_id"]
|
||||
row = db.execute("SELECT * FROM outreach_provider_configs WHERE organization_id=?", (org,)).fetchone()
|
||||
if payload is None:
|
||||
return self.send_json(200, self._provider_json(row, org))
|
||||
provider = str(payload.get("provider", row["provider"] if row else "")).strip().lower()
|
||||
if provider and provider not in self.OUTREACH_PROVIDERS: return self.send_json(400, {"error": "invalid_provider"})
|
||||
enabled = bool(payload.get("enabled", bool(row["enabled"]) if row else False))
|
||||
policy = payload.get("policy", payload.get("legal_policy", {} if not row else None))
|
||||
if policy is None:
|
||||
try: policy = json.loads(row["policy_json"] or "{}")
|
||||
except (TypeError, ValueError): policy = {"consent_required": True}
|
||||
if not isinstance(policy, dict) or len(policy) > 20: return self.send_json(400, {"error": "invalid_policy"})
|
||||
policy = {str(k)[:80]: v for k, v in policy.items()}
|
||||
policy.setdefault("consent_required", True)
|
||||
try:
|
||||
daily = int(payload.get("daily_cap", row["daily_cap"] if row else self.OUTREACH_DAILY_CAP)); batch = int(payload.get("batch_cap", row["batch_cap"] if row else self.OUTREACH_BATCH_CAP))
|
||||
except (TypeError, ValueError): return self.send_json(400, {"error": "invalid_limits"})
|
||||
if daily < 1 or daily > self.OUTREACH_DAILY_CAP or batch < 1 or batch > self.OUTREACH_BATCH_CAP: return self.send_json(400, {"error": "invalid_limits"})
|
||||
secret = payload.get("secret", None)
|
||||
fingerprint = row["secret_fingerprint"] if row else ""
|
||||
if secret is not None:
|
||||
if not isinstance(secret, str) or not secret or len(secret) > 4096: return self.send_json(400, {"error": "invalid_secret"})
|
||||
fingerprint = hashlib.sha256(secret.encode()).hexdigest()
|
||||
if enabled and (not provider or not fingerprint): return self.send_json(400, {"error": "provider_credentials_required"})
|
||||
if row:
|
||||
db.execute("UPDATE outreach_provider_configs SET provider=?,enabled=?,secret_fingerprint=?,policy_json=?,daily_cap=?,batch_cap=?,updated_at=CURRENT_TIMESTAMP WHERE organization_id=?", (provider, int(enabled), fingerprint, json.dumps(policy, sort_keys=True), daily, batch, org))
|
||||
else:
|
||||
db.execute("INSERT INTO outreach_provider_configs(organization_id,provider,enabled,secret_fingerprint,policy_json,daily_cap,batch_cap) VALUES(?,?,?,?,?,?,?)", (org, provider, int(enabled), fingerprint, json.dumps(policy, sort_keys=True), daily, batch))
|
||||
self.audit(db, user, "outreach.provider_config.updated", provider or "disabled"); db.commit()
|
||||
return self.send_json(200, self._provider_json(db.execute("SELECT * FROM outreach_provider_configs WHERE organization_id=?", (org,)).fetchone(), org))
|
||||
|
||||
def _draft_json(self, row):
|
||||
item = row_json(row)
|
||||
for field, default in (("template_json", {}), ("citations_json", []), ("provenance_json", {})):
|
||||
key = field[:-5]
|
||||
try: item[key] = json.loads(item.pop(field) or json.dumps(default))
|
||||
except (TypeError, ValueError): item[key] = default
|
||||
item["target_verified"] = bool(item.get("target_verified")); item["consent_confirmed"] = bool(item.get("consent_confirmed"))
|
||||
item["target"] = {"kind": item.pop("target_kind"), "value": item.pop("target_value"), "verified": item["target_verified"]}
|
||||
return item
|
||||
|
||||
def _template(self, text, business, evidence):
|
||||
pattern = re.compile(r"{{\s*([^}]+?)\s*}}")
|
||||
used = []
|
||||
def replace(match):
|
||||
token = match.group(1).strip()
|
||||
if token == "business.name": return str(business["name"])
|
||||
m = re.fullmatch(r"evidence\.(\d+)\.claim", token)
|
||||
if m:
|
||||
index = int(m.group(1));
|
||||
if index < 1 or index > len(evidence): raise ValueError("unsupported_template_variable")
|
||||
used.append(evidence[index - 1]["id"]); return str(evidence[index - 1]["claim"])
|
||||
raise ValueError("unsupported_template_variable")
|
||||
return pattern.sub(replace, text), sorted(set(used))
|
||||
|
||||
def _target_exists(self, db, bid, org, kind, value):
|
||||
if kind == "email":
|
||||
return bool(db.execute("SELECT id FROM businesses WHERE id=? AND organization_id=? AND email=?", (bid, org, value)).fetchone() or db.execute("SELECT id FROM contacts WHERE business_id=? AND organization_id=? AND email=?", (bid, org, value)).fetchone() or db.execute("SELECT id FROM contact_extractions WHERE business_id=? AND organization_id=? AND kind='email' AND value=?", (bid, org, value)).fetchone())
|
||||
return bool(db.execute("SELECT id FROM businesses WHERE id=? AND organization_id=? AND phone=?", (bid, org, value)).fetchone() or db.execute("SELECT id FROM contacts WHERE business_id=? AND organization_id=? AND phone=?", (bid, org, value)).fetchone() or db.execute("SELECT id FROM contact_extractions WHERE business_id=? AND organization_id=? AND kind IN ('phone','whatsapp') AND value=?", (bid, org, value)).fetchone())
|
||||
|
||||
def create_outreach_draft(self, bid, payload, db, user):
|
||||
org = user["organization_id"]; business = self.business(db, bid, org)
|
||||
if not business: return self.send_json(404, {"error": "not_found"})
|
||||
target = payload.get("target", {}); kind = str(target.get("kind", "email")).lower(); value = str(target.get("value", "")).strip().lower()
|
||||
if kind not in self.OUTREACH_KINDS or not value or len(value) > 320 or not isinstance(target.get("verified", False), bool): return self.send_json(400, {"error": "invalid_target"})
|
||||
subject, body = str(payload.get("subject", "")).strip(), str(payload.get("body", "")).strip()
|
||||
if not subject or not body or len(subject) > 500 or len(body) > 10000: return self.send_json(400, {"error": "invalid_draft"})
|
||||
if contains_secret(payload): return self.send_json(400, {"error": "secret_not_permitted"})
|
||||
try:
|
||||
evidence = [dict(r) for r in db.execute("SELECT id,kind,url,claim,created_at FROM evidence WHERE business_id=? AND organization_id=? ORDER BY id", (bid, org))]
|
||||
subject, sids = self._template(subject, business, evidence); body, bids = self._template(body, business, evidence); ids = sorted(set(sids + bids))
|
||||
except ValueError as exc: return self.send_json(400, {"error": str(exc)})
|
||||
key = str(payload.get("idempotency_key", "")).strip() or hashlib.sha256(json.dumps({"business_id": bid, "target": target, "subject": subject, "body": body}, sort_keys=True).encode()).hexdigest()
|
||||
if len(key) > 200: return self.send_json(400, {"error": "invalid_idempotency_key"})
|
||||
existing = db.execute("SELECT * FROM outreach_drafts WHERE organization_id=? AND idempotency_key=?", (org, key)).fetchone()
|
||||
if existing: return self.send_json(200, self._draft_json(existing))
|
||||
if db.execute("SELECT COUNT(*) FROM outreach_drafts WHERE organization_id=? AND created_at>=datetime('now','-1 day')", (org,)).fetchone()[0] >= self.OUTREACH_DAILY_CAP: return self.send_json(429, {"error": "outreach_daily_cap"})
|
||||
cur = db.execute("INSERT INTO outreach_drafts(organization_id,business_id,target_kind,target_value,target_verified,subject,body,template_json,citations_json,provenance_json,legal_basis,consent_confirmed,actor_user_id,idempotency_key) VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?)", (org, bid, kind, value, int(target["verified"]), subject, body, json.dumps({"variables": ids}, sort_keys=True), json.dumps(ids), json.dumps({str(x): {"type": "evidence", "id": x} for x in ids}, sort_keys=True), str(payload.get("legal_basis", ""))[:100], int(bool(payload.get("consent_confirmed", False))), user["id"], key))
|
||||
self.audit(db, user, "outreach_draft.created", str(cur.lastrowid)); db.commit()
|
||||
return self.send_json(201, self._draft_json(db.execute("SELECT * FROM outreach_drafts WHERE id=?", (cur.lastrowid,)).fetchone()))
|
||||
|
||||
def list_outreach_drafts(self, db, user, query):
|
||||
try: limit = int((query.get("page_size") or [self.OUTREACH_BATCH_CAP])[0]); offset = int((query.get("offset") or [0])[0])
|
||||
except (ValueError, TypeError): return self.send_json(400, {"error": "invalid_pagination"})
|
||||
if limit < 1 or limit > self.OUTREACH_BATCH_CAP or offset < 0: return self.send_json(400, {"error": "invalid_pagination"})
|
||||
rows = db.execute("SELECT * FROM outreach_drafts WHERE organization_id=? ORDER BY id DESC LIMIT ? OFFSET ?", (user["organization_id"], limit + 1, offset)).fetchall()
|
||||
return self.send_json(200, {"organization_id": user["organization_id"], "items": [self._draft_json(r) for r in rows[:limit]], "limit": limit, "offset": offset, "has_more": len(rows) > limit})
|
||||
|
||||
def update_outreach_draft(self, did, payload, db, user):
|
||||
row = db.execute("SELECT * FROM outreach_drafts WHERE id=? AND organization_id=?", (did, user["organization_id"])).fetchone()
|
||||
if not row: return self.send_json(404, {"error": "not_found"})
|
||||
if row["status"] in ("approved", "sent"): return self.send_json(409, {"error": "draft_locked"})
|
||||
fields, values = [], []
|
||||
subject, body = row["subject"], row["body"]
|
||||
if "subject" in payload: subject = str(payload["subject"]).strip()
|
||||
if "body" in payload: body = str(payload["body"]).strip()
|
||||
if ("subject" in payload and (not subject or len(subject) > 500)) or ("body" in payload and (not body or len(body) > 10000)): return self.send_json(400, {"error": "invalid_draft"})
|
||||
if "subject" in payload or "body" in payload:
|
||||
business = self.business(db, row["business_id"], user["organization_id"])
|
||||
evidence = [dict(r) for r in db.execute("SELECT id,kind,url,claim,created_at FROM evidence WHERE business_id=? AND organization_id=? ORDER BY id", (row["business_id"], user["organization_id"]))]
|
||||
try:
|
||||
subject, sids = self._template(subject, business, evidence); body, bids = self._template(body, business, evidence)
|
||||
except ValueError as exc: return self.send_json(400, {"error": str(exc)})
|
||||
ids = sorted(set(sids + bids))
|
||||
fields += ["subject=?", "body=?", "template_json=?", "citations_json=?", "provenance_json=?"]
|
||||
values += [subject, body, json.dumps({"variables": ids}, sort_keys=True), json.dumps(ids), json.dumps({str(x): {"type": "evidence", "id": x} for x in ids}, sort_keys=True)]
|
||||
if not fields: return self.send_json(400, {"error": "no_changes"})
|
||||
fields.append("updated_at=CURRENT_TIMESTAMP"); values += [did, user["organization_id"]]
|
||||
db.execute("UPDATE outreach_drafts SET " + ",".join(fields) + " WHERE id=? AND organization_id=?", values); self.audit(db, user, "outreach_draft.updated", str(did)); db.commit()
|
||||
return self.send_json(200, self._draft_json(db.execute("SELECT * FROM outreach_drafts WHERE id=?", (did,)).fetchone()))
|
||||
|
||||
def approve_outreach_draft(self, did, db, user):
|
||||
row = db.execute("SELECT * FROM outreach_drafts WHERE id=? AND organization_id=?", (did, user["organization_id"])).fetchone()
|
||||
if not row: return self.send_json(404, {"error": "not_found"})
|
||||
if row["status"] != "pending_review": return self.send_json(409, {"error": "draft_not_reviewable"})
|
||||
db.execute("UPDATE outreach_drafts SET status='approved',approved_by=?,approved_at=CURRENT_TIMESTAMP,updated_at=CURRENT_TIMESTAMP WHERE id=? AND organization_id=?", (user["id"], did, user["organization_id"])); self.audit(db, user, "outreach_draft.approved", str(did)); db.commit()
|
||||
return self.send_json(200, self._draft_json(db.execute("SELECT * FROM outreach_drafts WHERE id=?", (did,)).fetchone()))
|
||||
|
||||
def send_outreach_draft(self, did, db, user):
|
||||
org = user["organization_id"]; row = db.execute("SELECT * FROM outreach_drafts WHERE id=? AND organization_id=?", (did, org)).fetchone()
|
||||
if not row: return self.send_json(404, {"error": "not_found"})
|
||||
cfg = db.execute("SELECT * FROM outreach_provider_configs WHERE organization_id=?", (org,)).fetchone(); reasons = []
|
||||
if not cfg or not cfg["enabled"] or not cfg["provider"] or not cfg["secret_fingerprint"]: reasons.append("provider")
|
||||
business = self.business(db, row["business_id"], org)
|
||||
if not row["target_verified"] or not self._target_exists(db, row["business_id"], org, row["target_kind"], row["target_value"]): reasons.append("verified_target")
|
||||
suppressed = is_suppressed({"email": row["target_value"] if row["target_kind"] == "email" else "", "phone": row["target_value"] if row["target_kind"] != "email" else "", "website_domain": business["website_domain"] if business else ""}, [dict(r) for r in db.execute("SELECT kind,value FROM suppressions WHERE organization_id=? AND active=1", (org,))])
|
||||
if suppressed: reasons.append("suppressed")
|
||||
policy = {}
|
||||
if cfg:
|
||||
try: policy = json.loads(cfg["policy_json"] or "{}")
|
||||
except (TypeError, ValueError): policy = {"consent_required": True}
|
||||
if policy.get("consent_required", True) and (not row["consent_confirmed"] or not row["legal_basis"]): reasons.append("consent_or_legal_policy")
|
||||
if row["status"] != "approved": reasons.append("approved_draft")
|
||||
if cfg and db.execute("SELECT COUNT(*) FROM outreach_drafts WHERE organization_id=? AND status='sent' AND sent_at>=datetime('now','-1 day')", (org,)).fetchone()[0] >= cfg["daily_cap"]: reasons.append("daily_cap")
|
||||
if reasons:
|
||||
status = "not_configured" if "provider" in reasons and (not cfg or not cfg["enabled"] or not cfg["provider"] or not cfg["secret_fingerprint"]) else "blocked"; self.audit(db, user, "outreach_draft.send_blocked", f"{did}:{','.join(reasons)}"); db.commit()
|
||||
return self.send_json(409, {"status": status, "blocked_reasons": reasons, "network_send": False, "id": did})
|
||||
reasons.append("network_send_disabled"); self.audit(db, user, "outreach_draft.send_blocked", f"{did}:network_send_disabled"); db.commit()
|
||||
return self.send_json(409, {"status": "blocked", "blocked_reasons": reasons, "network_send": False, "id": did})
|
||||
|
||||
def suggest_ai(self, bid, payload, db, user):
|
||||
org = user["organization_id"]
|
||||
business = self.business(db, bid, org)
|
||||
@@ -394,6 +555,8 @@ class ApiHandler(BaseHTTPRequestHandler):
|
||||
if path=="/api/v1/interactions": return self.list_interactions(db,org,parse_qs(parsed.query))
|
||||
if path=="/api/v1/suppressions": return self.list_suppressions(db,org,parse_qs(parsed.query))
|
||||
if path=="/api/v1/ai-runs": return self.list_ai_runs(db,user,parse_qs(parsed.query))
|
||||
if path=="/api/v1/outreach/drafts": return self.list_outreach_drafts(db,user,parse_qs(parsed.query))
|
||||
if path=="/api/v1/outreach/provider-config": return self.provider_config(db,user)
|
||||
if path in ("/api/v1/reports/pipeline","/api/v1/reports/outcomes","/api/v1/reports/activity"): return self.report(db,org,path.rsplit('/',1)[1],parse_qs(parsed.query))
|
||||
if path.startswith("/api/v1/jobs/"): return self.get_job_route(db,org,path,parse_qs(parsed.query))
|
||||
if path.startswith("/api/v1/businesses/"):
|
||||
@@ -753,6 +916,7 @@ class ApiHandler(BaseHTTPRequestHandler):
|
||||
if user["role"] not in MUTATING_ROLES:return self.send_json(403,{"error":"forbidden"})
|
||||
payload=self.read_json(); org=user["organization_id"]
|
||||
if path=="/api/v1/saved-filters": return self.save_filter(payload,db,user)
|
||||
if path=="/api/v1/outreach/provider-config": return self.provider_config(db,user,payload)
|
||||
if path=="/api/v1/businesses/bulk-review": return self.bulk_review(payload,db,user)
|
||||
bits_ai=path.split("/")
|
||||
if len(bits_ai)==7 and bits_ai[:4]==["","api","v1","businesses"] and bits_ai[5]=="ai" and bits_ai[6]=="suggest": return self.suggest_ai(int(bits_ai[4]) if bits_ai[4].isdigit() else -1,payload,db,user)
|
||||
@@ -762,6 +926,9 @@ class ApiHandler(BaseHTTPRequestHandler):
|
||||
if path.startswith("/api/v1/jobs/"):
|
||||
return self.job_action(db,user,path)
|
||||
if path=="/api/v1/businesses":return self.create_business(payload,db,user)
|
||||
if len(path.split("/"))==7 and path.split("/")[3:6]==["businesses",path.split("/")[4],"outreach"] and path.split("/")[6]=="drafts": return self.create_outreach_draft(int(path.split("/")[4]) if path.split("/")[4].isdigit() else -1,payload,db,user)
|
||||
bits_outreach=path.split("/")
|
||||
if len(bits_outreach)==7 and bits_outreach[:4]==["","api","v1","outreach"] and bits_outreach[4]=="drafts" and bits_outreach[5].isdigit() and bits_outreach[6] in {"approve","send"}: return self.approve_outreach_draft(int(bits_outreach[5]),db,user) if bits_outreach[6]=="approve" else self.send_outreach_draft(int(bits_outreach[5]),db,user)
|
||||
if path=="/api/v1/sources":return self.create_source(payload,db,user)
|
||||
if path=="/api/v1/discovery-queries":return self.create_query(payload,db,user)
|
||||
if path=="/api/v1/suppressions":return self.create_suppression(payload,db,user)
|
||||
@@ -802,7 +969,9 @@ class ApiHandler(BaseHTTPRequestHandler):
|
||||
if len(bits)==5 and bits[:4]==["","api","v1","interactions"] and bits[4].isdigit(): return self.update_interaction(int(bits[4]),self.read_json(),db,user)
|
||||
if len(bits)==5 and bits[:4]==["","api","v1","saved-filters"] and bits[4].isdigit(): return self.update_saved_filter(int(bits[4]),self.read_json(),db,user)
|
||||
if len(bits)==5 and bits[:4]==["","api","v1","score-rules"] and bits[4].isdigit(): return self.update_score_rule(int(bits[4]),self.read_json(),db,user)
|
||||
if len(bits)==5 and bits[:4]==["","api","v1","outreach"] and bits[4]=="provider-config": return self.provider_config(db,user,self.read_json())
|
||||
if len(bits)==5 and bits[:4]==["","api","v1","sources"] and bits[4].isdigit(): return self.update_source(int(bits[4]),self.read_json(),db,user)
|
||||
if len(bits)==6 and bits[:4]==["","api","v1","outreach"] and bits[4]=="drafts" and bits[5].isdigit(): return self.update_outreach_draft(int(bits[5]),self.read_json(),db,user)
|
||||
if len(bits)==6 and bits[:4]==["","api","v1","businesses"] and bits[5]=="pipeline":return self.update_pipeline(int(bits[4]) if bits[4].isdigit() else -1,self.read_json(),db,user)
|
||||
return self.send_json(404,{"error":"not_found"})
|
||||
finally:db.close()
|
||||
|
||||
@@ -280,3 +280,36 @@ CREATE TABLE IF NOT EXISTS ai_suggestions (
|
||||
created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_ai_suggestions_run ON ai_suggestions(organization_id,ai_run_id,id);
|
||||
|
||||
-- Phase 14 outreach preparation. Provider configuration is metadata plus a
|
||||
-- one-way secret fingerprint; outbound transport is intentionally disabled.
|
||||
CREATE TABLE IF NOT EXISTS outreach_provider_configs (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
organization_id TEXT NOT NULL REFERENCES organizations(id) ON DELETE CASCADE,
|
||||
provider TEXT NOT NULL DEFAULT '', enabled INTEGER NOT NULL DEFAULT 0,
|
||||
secret_fingerprint TEXT NOT NULL DEFAULT '',
|
||||
policy_json TEXT NOT NULL DEFAULT '{"consent_required":true}',
|
||||
daily_cap INTEGER NOT NULL DEFAULT 100, batch_cap INTEGER NOT NULL DEFAULT 25,
|
||||
created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP, updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
UNIQUE(organization_id)
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS outreach_drafts (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
organization_id TEXT NOT NULL REFERENCES organizations(id) ON DELETE CASCADE,
|
||||
business_id INTEGER NOT NULL REFERENCES businesses(id) ON DELETE CASCADE,
|
||||
target_kind TEXT NOT NULL CHECK(target_kind IN ('email','phone','whatsapp')),
|
||||
target_value TEXT NOT NULL,
|
||||
target_verified INTEGER NOT NULL DEFAULT 0,
|
||||
subject TEXT NOT NULL DEFAULT '', body TEXT NOT NULL DEFAULT '',
|
||||
template_json TEXT NOT NULL DEFAULT '{}', citations_json TEXT NOT NULL DEFAULT '[]',
|
||||
provenance_json TEXT NOT NULL DEFAULT '{}', status TEXT NOT NULL DEFAULT 'pending_review'
|
||||
CHECK(status IN ('pending_review','approved','blocked','sent')),
|
||||
legal_basis TEXT NOT NULL DEFAULT '', consent_confirmed INTEGER NOT NULL DEFAULT 0,
|
||||
actor_user_id INTEGER REFERENCES users(id) ON DELETE SET NULL,
|
||||
approved_by INTEGER REFERENCES users(id) ON DELETE SET NULL, approved_at TEXT,
|
||||
sent_at TEXT, idempotency_key TEXT, created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
UNIQUE(organization_id,idempotency_key)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_outreach_drafts_org ON outreach_drafts(organization_id,created_at DESC,id DESC);
|
||||
CREATE INDEX IF NOT EXISTS idx_outreach_drafts_business ON outreach_drafts(organization_id,business_id,id DESC);
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
import json
|
||||
import os
|
||||
import sqlite3
|
||||
import threading
|
||||
import unittest
|
||||
from http.client import HTTPConnection
|
||||
from tempfile import TemporaryDirectory
|
||||
|
||||
from app.main import create_server, hash_password
|
||||
|
||||
|
||||
class Phase14ApiTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = TemporaryDirectory()
|
||||
self.old = {k: os.environ.get(k) for k in ("BOOTSTRAP_ADMIN_EMAIL", "BOOTSTRAP_ADMIN_PASSWORD")}
|
||||
os.environ["BOOTSTRAP_ADMIN_EMAIL"] = "outreach-owner@example.test"
|
||||
os.environ["BOOTSTRAP_ADMIN_PASSWORD"] = "outreach-password"
|
||||
self.db_path = self.tmp.name + "/outreach.db"
|
||||
self.server = create_server("127.0.0.1", 0, self.db_path)
|
||||
self.thread = threading.Thread(target=self.server.serve_forever, daemon=True); self.thread.start()
|
||||
self.conn = HTTPConnection("127.0.0.1", self.server.server_port, timeout=3); self.cookie = None
|
||||
self.request("POST", "/api/v1/auth/login", {"email": "outreach-owner@example.test", "password": "outreach-password"})
|
||||
|
||||
def tearDown(self):
|
||||
self.server.shutdown(); self.server.server_close(); self.thread.join(timeout=2); self.tmp.cleanup()
|
||||
for k, v in self.old.items():
|
||||
if v is None: os.environ.pop(k, None)
|
||||
else: os.environ[k] = v
|
||||
|
||||
def request(self, method, path, payload=None):
|
||||
body = json.dumps(payload).encode() if payload is not None else None
|
||||
headers = {"Content-Type": "application/json"} if body else {}
|
||||
if self.cookie: headers["Cookie"] = self.cookie
|
||||
self.conn.request(method, path, body, headers); response = self.conn.getresponse()
|
||||
if response.getheader("Set-Cookie"): self.cookie = response.getheader("Set-Cookie").split(";", 1)[0]
|
||||
return response.status, json.loads(response.read() or b"{}")
|
||||
|
||||
def business(self, email="target@example.test"):
|
||||
status, item = self.request("POST", "/api/v1/businesses", {"name": "Target Co", "website": "https://target.test", "email": email})
|
||||
self.assertEqual(status, 201); return item
|
||||
|
||||
def test_draft_lifecycle_is_evidence_grounded_and_audited(self):
|
||||
b = self.business(); bid = b["id"]
|
||||
self.request("POST", f"/api/v1/businesses/{bid}/evidence", {"kind": "source", "url": "https://target.test/about", "claim": "Makes solar panels"})
|
||||
status, draft = self.request("POST", f"/api/v1/businesses/{bid}/outreach/drafts", {"target": {"kind": "email", "value": "target@example.test", "verified": True}, "subject": "Hello {{business.name}}", "body": "We saw: {{evidence.1.claim}}", "idempotency_key": "draft-1"})
|
||||
self.assertEqual(status, 201); self.assertEqual(draft["status"], "pending_review"); self.assertIn("Target Co", draft["subject"]); self.assertIn("Makes solar panels", draft["body"])
|
||||
self.assertEqual(len(draft["citations"]), 1); self.assertEqual(self.request("POST", f"/api/v1/businesses/{bid}/outreach/drafts", {"target": {"kind": "email", "value": "target@example.test", "verified": True}, "subject": "ignored", "body": "ignored", "idempotency_key": "draft-1"})[0], 200)
|
||||
self.assertEqual(self.request("PATCH", f"/api/v1/outreach/drafts/{draft['id']}", {"body": "Updated {{evidence.1.claim}}"})[0], 200)
|
||||
status, approved = self.request("POST", f"/api/v1/outreach/drafts/{draft['id']}/approve", {}); self.assertEqual(status, 200); self.assertEqual(approved["status"], "approved")
|
||||
db = sqlite3.connect(self.db_path); actions = [r[0] for r in db.execute("SELECT action FROM audit_log")]; db.close()
|
||||
self.assertTrue({"outreach_draft.created", "outreach_draft.updated", "outreach_draft.approved"}.issubset(actions))
|
||||
|
||||
def test_send_is_not_configured_and_all_gates_are_reported(self):
|
||||
b = self.business(); bid = b["id"]
|
||||
_, d = self.request("POST", f"/api/v1/businesses/{bid}/outreach/drafts", {"target": {"kind": "email", "value": "target@example.test", "verified": False}, "subject": "Hi", "body": "Body"})
|
||||
status, result = self.request("POST", f"/api/v1/outreach/drafts/{d['id']}/send", {}); self.assertEqual(status, 409); self.assertEqual(result["status"], "not_configured"); self.assertIn("provider", result["blocked_reasons"])
|
||||
|
||||
def test_suppression_and_unapproved_target_block_send_and_no_network(self):
|
||||
b = self.business(); bid = b["id"]
|
||||
self.request("PATCH", "/api/v1/outreach/provider-config", {"provider": "smtp", "enabled": True, "secret": "test-secret", "legal_policy": {"consent_required": False}})
|
||||
self.request("POST", "/api/v1/suppressions", {"kind": "email", "value": "target@example.test"})
|
||||
_, d = self.request("POST", f"/api/v1/businesses/{bid}/outreach/drafts", {"target": {"kind": "email", "value": "target@example.test", "verified": True}, "subject": "Hi", "body": "Body"})
|
||||
self.request("POST", f"/api/v1/outreach/drafts/{d['id']}/approve", {})
|
||||
status, result = self.request("POST", f"/api/v1/outreach/drafts/{d['id']}/send", {}); self.assertEqual(status, 409); self.assertEqual(result["status"], "blocked"); self.assertIn("suppressed", result["blocked_reasons"]); self.assertFalse(result["network_send"])
|
||||
|
||||
def test_provider_config_never_returns_secret_and_tenant_isolation(self):
|
||||
status, config = self.request("PATCH", "/api/v1/outreach/provider-config", {"provider": "smtp", "enabled": True, "secret": "super-secret", "legal_policy": {"consent_required": True}})
|
||||
self.assertEqual(status, 200); self.assertNotIn("secret", json.dumps(config).lower()); self.assertTrue(config["enabled"])
|
||||
self.assertNotIn("super-secret", json.dumps(config))
|
||||
ph, salt = hash_password("other-password"); db = sqlite3.connect(self.db_path); db.execute("INSERT INTO organizations(id,name) VALUES('other-tenant','Other')"); db.execute("INSERT INTO users(organization_id,email,password_hash,password_salt,role) VALUES(?,?,?,?,?)", ("other-tenant", "other@example.test", ph, salt, "owner")); db.commit(); db.close()
|
||||
self.cookie = None; self.assertEqual(self.request("POST", "/api/v1/auth/login", {"email": "other@example.test", "password": "other-password"})[0], 200)
|
||||
self.assertEqual(self.request("GET", "/api/v1/outreach/provider-config")[1]["enabled"], False)
|
||||
self.assertEqual(self.request("GET", "/api/v1/outreach/drafts")[1]["items"], [])
|
||||
|
||||
def test_limits_and_unknown_template_variable(self):
|
||||
b = self.business(); bid = b["id"]
|
||||
status, result = self.request("POST", f"/api/v1/businesses/{bid}/outreach/drafts", {"target": {"kind": "email", "value": "target@example.test", "verified": True}, "subject": "Hi {{not.evidence}}", "body": "Body"})
|
||||
self.assertEqual(status, 400); self.assertEqual(result["error"], "unsupported_template_variable")
|
||||
self.assertEqual(self.request("GET", "/api/v1/outreach/drafts?page_size=101")[0], 400)
|
||||
|
||||
|
||||
if __name__ == "__main__": unittest.main()
|
||||
Reference in New Issue
Block a user