root/MarketingTool
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

deploy Maps-backed ProspectOS workspace

Browse Source
This commit is contained in:
Marco0300
2026-09-05 15:47:02 +02:00
parent 03d545532a
commit 6dd9531a66
8 changed files with 929 additions and 157 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+74 -27
View File
@@ -1,45 +1,92 @@
# Browser-only ProspectOS
# ProspectOS · Maps Scout
This is the stripped-down replacement for the larger ProspectOS application.
It searches public engines (Google, Bing, DuckDuckGo, Brave Search, and
Startpage) through a headless Playwright Chromium browser, moving to the next
engine when a server-side challenge, timeout, or empty/changed result page is
encountered. Each
discovery run builds a query from category, keywords, and location, filters
results already present in the local catalogue by root domain, and stores only
new prospects. The explorer supports search and human review states
(`pending`, `approved`, `rejected`, `contacted`).
A small, personal-use prospect workspace. It sends one search at a time to
the bundled [gosom/google-maps-scraper](https://github.com/gosom/google-maps-scraper)
worker, waits for its asynchronous job to finish, filters places already in
the local catalogue, and presents the remaining places for human review.
There is deliberately no source registry, email scraper, AI provider, CRM,
outreach sender, or automatic contact action.
The interface is intentionally focused: discovery, run progress, catalogue
search, and review status. It does not send outreach, create a CRM campaign, or
make an AI decision. The opportunity score is a transparent triage hint based
on fields returned by Maps (for example, a missing website and a listed phone
number); you make the final review decision.
## Run with Docker
## Start the stack
```bash
cp .env.example .env
docker compose up -d --build
```
Open `http://SERVER:8000`.
Open `http://SERVER:8000` for ProspectOS. The Maps worker is available on
`http://SERVER:8081` for its one-time setup.
If another service already owns port 8000, choose the host port without
changing the container:
### Connect the Maps worker
1. Open port `8081` in a browser on your private network.
2. Use the worker's admin screen to create an API key for this local app.
3. Put that key in `.env` as `GMAPS_API_KEY=...`.
4. Restart the app with `docker compose up -d --build` and reload ProspectOS.
The app calls `/api/v1/scrape` to queue a job and `/api/v1/jobs/{id}` to poll
it. Jobs are deliberately asynchronous because a Maps scrape can take several
minutes. A run that loses its worker connection is shown as failed with a
specific error rather than remaining stuck forever.
## Configuration
| Variable | Default | Purpose |
|---|---:|---|
| `HOST_PORT` | `8000` | Host port for ProspectOS |
| `GMAPS_HOST_PORT` | `8081` | Host port for the Maps worker admin UI |
| `GMAPS_API_KEY` | empty | API key created in the Maps worker admin UI |
| `GMAPS_REQUEST_TIMEOUT` | `300` | Per-job scraper timeout in seconds |
| `GMAPS_POLL_SECONDS` | `2` | Poll interval while a job is running |
| `DISCOVERY_BACKEND` | `gmaps` when `GMAPS_URL` is set | Set to `browser` only for the old public-search fallback |
The app stores its catalogue in the `prospect_data` Docker volume. The worker
stores its own job data in `gmaps_data`; keep both volumes if you want to retain
history. To inspect status:
```bash
HOST_PORT=8080 docker compose up -d --build
docker compose ps
docker compose logs --tail=100 app
docker compose logs --tail=100 gmaps
```
If an engine presents a CAPTCHA or bot challenge, the run moves on to the next
engine. The app does not bypass challenges or rotate identities. Keep runs
small and spaced out. The implementation uses a fresh headless browser
context for each engine and unwraps normal search redirect links before
deduplication. If every public engine blocks the server IP, the run reports a
clear `search_engines_unavailable` error instead of inventing prospects.
## Local fallback mode
## Run locally
For a quick local smoke test without the Maps worker:
```bash
python -m pip install -r api/requirements.txt
PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH=/usr/bin/chromium python api/server.py
DISCOVERY_BACKEND=browser GMAPS_URL= docker compose up -d --build
```
The browser executable path can be changed for the host operating system.
This uses Playwright against public search engines, keeps the same catalogue
and review UI, and does not bypass CAPTCHAs or other access controls.
## Data and review behavior
Each saved place includes its name, Maps link, website (when present), root
domain, address, phone, category, rating, review count, public emails returned
by the worker when explicitly enabled, source ID, and raw result JSON. A place
with no website is still retained using a stable `maps:<place_id>` catalogue
key, so it can be reviewed as a potential website opportunity. Existing
websites are deduplicated by root domain.
Email extraction is off by default in the UI. If you enable it for a run, the
worker may visit the public business website; expect a longer run and review
the resulting provenance before using any contact information.
## Tests
```bash
python3 -m py_compile api/server.py
node --check web/app.js
python3 -m unittest discover -s tests -q
```
This project is packaged for personal research. Public availability does not
override a website's terms, robots controls, or applicable privacy law; keep
searches small, use the worker only on data you are permitted to collect, and
do not turn the review screen into unattended outreach.