feat: add self-hosted SearXNG research mode
CI / compose (push) Successful in 13m39s

This commit is contained in:
Marco0300
2026-09-03 21:49:02 +02:00
parent 0f70674b16
commit d298a98723
11 changed files with 175 additions and 42 deletions
+17 -14
View File
@@ -17,8 +17,7 @@ After the first successful login, remove both values from `.env`, restart the AP
Copy `.env.example` to an untracked deployment environment file. Production requires a secret-manager supplied `SESSION_SECRET` of at least 32 characters and refuses `AUTOMATED_OUTREACH_ENABLED=true`. Keep bootstrap credentials one-time only; remove and rotate them after provisioning. Never place secrets in images, Compose YAML, logs, backups, or public web roots.
For optional native Nous Portal Chat Completions tool-calling discovery, set these
server-side variables:
For criteria-first AI web research without a paid scraper, use the self-hosted mode:
```dotenv
AI_RESEARCH_PROVIDER=nous_portal
@@ -26,20 +25,24 @@ NOUS_API_KEY=<Nous Portal API key>
NOUS_MODEL=Hermes-4-405B
NOUS_BASE_URL=https://inference-api.nousresearch.com/v1
NOUS_ALLOWED_HOSTS=inference-api.nousresearch.com
FIRECRAWL_API_KEY=<Firecrawl API key>
FIRECRAWL_BASE_URL=https://api.firecrawl.dev/v2
FIRECRAWL_ALLOWED_HOSTS=api.firecrawl.dev
SEARXNG_BASE_URL=http://searxng:8080
SEARXNG_ALLOWED_HOSTS=searxng
SEARXNG_SECRET_KEY=<random secret>
```
The adapter calls Nous at `/chat/completions` with strict `web_search` and
`scrape_website` function tools. Tool calls are executed only against the
allowlisted Firecrawl-compatible API, capped at 4 calls and 16 KiB per tool
result. Prompt-injection-shaped criteria are rejected and tool/page content is
untrusted data. The final model response is parsed only as structured JSON
HTTPS targets; the existing crawler performs SSRF validation and persists
fetched-page evidence. Missing either key, unavailable providers, unsafe base
URLs, malformed tool calls, oversized responses, and exhausted budgets fail
closed. Status metadata never includes secrets.
Compose starts SearXNG without publishing a host port. The API reaches it only
on the internal Compose network; SearXNG alone has a separate egress network for
its upstream search engines. `web_search` calls SearXNG's JSON endpoint and
`scrape_website` uses the API's bounded SSRF-safe scanner. No Firecrawl key is
required. Firecrawl variables remain an optional legacy compatibility fallback.
The Nous adapter calls `/chat/completions` with strict `web_search` and
`scrape_website` function tools. Tool calls are capped at 4 and each result at
16 KiB; criteria, model output, page text, redirects, and candidate URLs remain
bounded and untrusted. Only structured HTTPS targets are accepted and the
existing crawler fetches/persists evidence. Missing/unsafe configuration,
unavailable providers, malformed calls, oversized responses, SSRF targets, and
exhausted budgets fail closed. Status metadata never includes secrets.
The prior OpenAI Responses and generic provider variables remain supported only
as compatibility adapters.