MarketingTool/README.md

# ProspectOS · Maps Scout

A small, personal-use prospect workspace. It sends one search at a time to
the bundled [gosom/google-maps-scraper](https://github.com/gosom/google-maps-scraper)
worker, waits for its asynchronous job to finish, filters places already in
the local catalogue, and presents the remaining places for human review.

The interface is split into Dashboard, Discover, Prospects, Runs, Logs, and Settings
views. It includes website health and email evidence, explainable scoring,
domain ideas, suppression, CSV export, scheduled runs, rescans, and human review. It never sends outreach
automatically. The Prospects explorer has an **All high potential** tab and
category tabs populated from each discovery run, with every tab sorted from
highest to lowest opportunity score.

Optional AI can analyse a prospect and draft a subject/email from supplied
evidence and your offer; you edit and approve the draft yourself. Drafts now
also return the reasoning, stored evidence IDs used, facts that still need
confirmation, and a follow-up review note so a polished email is not mistaken
for verified information.
The **Logs** view separates API calls, user actions, AI requests, searches,
Browser Use/tool steps, and errors. AI request payloads and responses are
truncated and credentials such as API keys, authorization headers, tokens, and
passwords are redacted. Logs are retained up to `LOG_RETENTION` entries (20,000
by default).
From a prospect's detail drawer, **Assess with AI** stores a contact probability,
recommendation, and short evidence-grounded rationale for that specific company.
The same drawer can optionally run a bounded Browser Use investigation to look
for a likely official website when Maps did not list one. Candidate domains are
shown with evidence and must be accepted by you before they replace the
catalogue record.

## Start the stack

```bash
cp .env.example .env
docker compose up -d --build
```

Open `http://SERVER:8000` for ProspectOS. The Maps worker is available on
`http://SERVER:8081` if you want to inspect its native Web UI.

### How the worker connects

No Google Cloud API key is required. The bundled standard worker exposes a
local REST API on the Docker network, and ProspectOS calls it directly.

The app uses `/api/v1/jobs` to queue a search, polls `/api/v1/jobs/{id}`, and
downloads `/api/v1/jobs/{id}/download` when the job finishes. Jobs are
deliberately asynchronous because a Maps scrape can take several minutes.

Only set `GMAPS_API_MODE=saas` and `GMAPS_API_KEY=...` if you deliberately
replace the bundled worker with gosom's optional SaaS edition.

A run that loses its worker connection is shown as failed with a specific
error rather than remaining stuck forever.

## Configuration

| Variable | Default | Purpose |
|---|---:|---|
| `HOST_PORT` | `8000` | Host port for ProspectOS |
| `GMAPS_HOST_PORT` | `8081` | Host port for the Maps worker admin UI |
| `GMAPS_API_MODE` | `standard` | `standard` for the bundled worker; `saas` only for the optional SaaS API |
| `GMAPS_API_KEY` | empty | Not required in standard mode; only used in SaaS mode |
| `GMAPS_REQUEST_TIMEOUT` | `900` | Maximum wait for the Maps worker in seconds; raise it for very large searches |
| `GMAPS_POLL_SECONDS` | `2` | Poll interval while a job is running |
| `OUTBOUND_USER_AGENT` | `ProspectOS/2.0 (+personal-use)` | Identifies provider HTTP requests; avoids Python's default client fingerprint |
| `DISCOVERY_BACKEND` | `gmaps` when `GMAPS_URL` is set | Set to `browser` only for the bounded public-search fallback |
| `BROWSER_USE_ENABLED` | `true` | Enables the optional website-candidate investigator |
| `BROWSER_USE_MAX_STEPS` | `10` | Maximum Browser Use actions for one investigation (3–50); can be changed in Settings |
| `BROWSER_USE_TIMEOUT_SECONDS` | `600` | Hard timeout for one investigation (30–1800 seconds); can be changed in Settings |
| `BROWSER_USE_LLM_TIMEOUT_SECONDS` | `90` | Timeout for one Browser Use model call (10–300 seconds); can be changed in Settings |
| `WEBSITE_SCAN_TIMEOUT` | `8` | Per-page public website/email scan timeout (2–60 seconds); can be changed in Settings |
| `ANONYMIZED_TELEMETRY` | `false` | Keeps Browser Use telemetry disabled in this personal deployment |
| `LOG_RETENTION` | `20000` | Maximum operational log entries retained in SQLite |
| `AI_PROVIDER` | `custom` | Set to `nous` for the Nous Research Portal, or `custom` for another OpenAI-compatible endpoint |
| `AI_BASE_URL` | empty | Optional OpenAI-compatible `/chat/completions` endpoint |
| `AI_API_KEY` | empty | Optional provider key; keep server-side |
| `AI_MODEL` | empty | Model name used for evidence-grounded drafts |

### Nous Research Portal

Choose **Nous Portal · free models** in Settings, paste a Nous API key, select a
free model, save, and use **Test connection** before drafting a message. The
app uses `https://inference-api.nousresearch.com/v1` automatically and tries
the live `/models` catalog; if the catalog is unavailable it shows a stable
fallback list of the free models currently published by Nous. Availability and
model IDs can change, so refresh Settings periodically.

If a Nous request returns `ai_cloudflare_1010`, rebuild the app from the latest
source so provider requests include the explicit `ProspectOS/2.0` client
identity. The old build used Python's default `Python-urllib` fingerprint, which
can be rejected by the Portal's Cloudflare policy. This is not a Maps error and
it cannot be fixed by changing the model name. If the rebuilt app still receives
1010, test the same key from the server with the provider's documented
OpenAI-compatible endpoint and contact Nous support if the endpoint continues to
block that client; do not try to evade the security challenge.

The bundled fallback entries are LongCat 2.0, Solar Pro 4, Step 3.7 Flash,
Laguna S 2.1, Laguna XS 2.1, Ling 3.0 Flash Fin, and Ling 3.0 Flash Sante
(using the Portal's `:free` model suffix).

Hermes Agent has its own browser-based OAuth flow, but that flow stores a
Hermes refresh token and is not a portable credential for this standalone
Docker app. ProspectOS therefore uses a server-side Nous API key here. The key
is never returned by the Settings API or sent to the browser after saving.

### Browser Use website investigator

Browser Use is an optional local browser agent. It uses the AI provider already
configured in Settings to navigate a small number of public search/result pages,
then returns candidate URLs as structured data. It is intentionally limited to
unauthenticated research: no proxy rotation, stealth fingerprints, CAPTCHA
solving, login, or challenge bypass is configured. If a site blocks the browser,
the action returns an explicit error and the rest of the prospect workflow is
left intact. Browser Use itself is open source, but each investigation still
uses your configured model/provider and may consume its quota.

To use it:

1. Configure and test an AI provider in **Settings** (a Nous key plus a selected
   model also works).
2. Open a prospect that has no detected website and choose **Find possible
   websites**.
3. Review the returned domains and evidence. Choose **Accept** only when the
   domain is genuinely the business's official site; accepting runs the normal
   website health and public-email scan.

The API equivalents are `POST /api/prospects/{id}/website-candidates`,
`GET /api/prospects/{id}/website-candidates`, and
`PATCH /api/prospects/{id}/website-candidates/{candidate_id}` with
`{"status":"accepted"}` or `{"status":"rejected"}`.

### Runtime settings

Open **Settings → Browser and discovery timing** to change Browser Use's
enabled state, total investigation timeout, maximum agent steps, and per-call
AI timeout. The same panel controls the Maps worker wait timeout/poll interval,
search depth, and website health/email scan timeout. Values are stored in the
local SQLite database, survive container restarts, and override `.env` defaults
for new runs. The safe UI ranges prevent a malformed value from creating an
unbounded worker.

The app stores its catalogue in the `prospect_data` Docker volume. The worker
stores its own job data in `gmaps_data`; keep both volumes if you want to retain
history. To inspect status:

```bash
docker compose ps
docker compose logs --tail=100 app
docker compose logs --tail=100 gmaps
```

## Local fallback mode

For a quick local smoke test without the Maps worker:

```bash
DISCOVERY_BACKEND=browser GMAPS_URL= docker compose up -d --build
```

This uses Playwright against public search engines, keeps the same catalogue
and review UI, and does not bypass CAPTCHAs or other access controls.

## Data and review behavior

Each saved place includes its name, Maps link, website (when present), root
domain, address, phone, category, rating, review count, public emails returned
by the worker when available, source ID, and raw result JSON. A place
with no website is still retained using a stable `maps:<place_id>` catalogue
key, so it can be reviewed as a potential website opportunity. Existing
websites are deduplicated by root domain.

Email extraction is always enabled. The worker may visit each public business
website found by Maps; expect longer runs and review the resulting provenance
before using any contact information. It does not invent email addresses when
none are published.

Website scans are bounded and SSRF-safe. A scan records HTTP status, final URL,
latency, HTTPS/TLS, title, technology hints, contact-page emails, and errors.
Use `/api/prospects/{id}/scan` for a manual rescan. The explorer's “Draft AI
email” action calls `/api/prospects/{id}/message`, and **Assess with AI** calls
`/api/prospects/{id}/assess`; both require the optional AI
variables above and accepts your exact offer/instructions (for example, the
Gold Plus R119/month plan and a free rebuild). Suppressions are enforced before
new records are saved and can be managed through `/api/suppressions`.

## Tests

```bash
python3 -m py_compile api/server.py
node --check web/app.js
python3 -m unittest discover -s tests -q
```

This project is packaged for personal research. Public availability does not
override a website's terms, robots controls, or applicable privacy law; keep
searches small, use the worker only on data you are permitted to collect, and
do not turn the review screen into unattended outreach.