sakshamandClaude Sonnet 5
1be3e53584
Expands the Configuration section to explain why (nothing paid is ever committed, .env is gitignored) and separates the two independent places to add real keys once you have your own: .env for deployment- wide provider selection/defaults, vs. the Settings page's "Your API keys" (per-user) and admin-only "Server secrets" (Turnstile, Resend) for a running deployment with no restart needed. Also links DEPLOYMENT.md and the live ciagent.org/git.ciagent.org reference deployment, and fixes the Celery queue list to include "enrichment". Co-Authored-By: Claude Sonnet 5 <[email protected]>
110 lines
6.7 KiB
Markdown
110 lines
6.7 KiB
Markdown
# CI Agent — Competitive Intelligence Monitoring
|
|
|
|
CI Agent monitors companies you choose, collects publicly available information from multiple sources on a schedule, analyzes it with an LLM into an evidence-linked report, detects meaningful changes between runs, and alerts you by email (and optionally SMS) with a severity and confidence score.
|
|
|
|
See [`PLAN.md`](PLAN.md) for the build strategy, [`ARCHITECTURE.md`](ARCHITECTURE.md) for system design, [`SECURITY.md`](SECURITY.md) for the threat model, [`TASKS.md`](TASKS.md) for the live implementation checklist, [`KNOWN_LIMITATIONS.md`](KNOWN_LIMITATIONS.md) for what's stubbed vs. fully live, [`DEPLOYMENT.md`](DEPLOYMENT.md) for running this in production behind Cloudflare/Nginx, and [`docs/FIREBASE_MIGRATION.md`](docs/FIREBASE_MIGRATION.md) for a (skeptical) look at what moving to Firebase would take.
|
|
|
|
A live reference deployment runs at [ciagent.org](https://ciagent.org), with its own code hosted on a self-hosted Gitea instance at [git.ciagent.org](https://git.ciagent.org) (public read, admin-only write) — click "Git Repository" on the landing page.
|
|
|
|
## Quick start (Docker)
|
|
|
|
```bash
|
|
git clone <repository>
|
|
cd ci-agent
|
|
cp .env.example .env
|
|
docker compose up --build
|
|
```
|
|
|
|
Then visit:
|
|
|
|
| Service | URL |
|
|
|---|---|
|
|
| Frontend | http://localhost:3000 |
|
|
| Backend API | http://localhost:8000 |
|
|
| API docs (Swagger) | http://localhost:8000/docs |
|
|
|
|
The default `.env.example` runs in `AUTH_MODE=local` (no login required, fixed dev user) with `LLM_PROVIDER=mock` and `SEARCH_PROVIDER=mock` — the whole demo workflow works with zero paid API keys.
|
|
|
|
`api`, `worker`, and `beat` build from the same Dockerfile but are separate images — after adding a Python dependency, run `docker compose build api worker beat` (not just `restart`) or they'll crash with `ModuleNotFoundError` on stale images.
|
|
|
|
Similarly, after adding an npm dependency to `apps/web`, a plain rebuild isn't enough either — `web`'s `node_modules` *and* `.next` are both persistent anonymous Docker volumes that survive `docker compose build web` and even a plain `docker compose restart web`. This means a new package can still 404, and — on Windows + Docker Desktop — an edited existing file can keep serving its pre-edit output after a restart, since the on-disk `.next` build cache isn't cleared by a restart. Run `docker compose rm -f -s -v web && docker compose up -d web` to actually pick up new packages or force a clean recompile.
|
|
|
|
## Running without Docker
|
|
|
|
**Backend:**
|
|
|
|
```bash
|
|
cd apps/api
|
|
python -m venv .venv
|
|
./.venv/Scripts/activate # or `source .venv/bin/activate` on macOS/Linux
|
|
pip install -e ".[dev]"
|
|
cp ../../.env.example ../../.env # edit DATABASE_URL to the sqlite line if you don't have Postgres running
|
|
alembic upgrade head
|
|
uvicorn app.main:app --reload
|
|
```
|
|
|
|
**Frontend:**
|
|
|
|
```bash
|
|
cd apps/web
|
|
npm install
|
|
npm run dev
|
|
```
|
|
|
|
## Common commands
|
|
|
|
```bash
|
|
# Database migrations
|
|
cd apps/api && alembic upgrade head
|
|
cd apps/api && alembic revision --autogenerate -m "description"
|
|
|
|
# Run a Celery worker + beat (only needed outside Docker)
|
|
cd apps/api && celery -A app.tasks.celery_app worker --loglevel=INFO -Q default,collection,analysis,notifications,maintenance,enrichment
|
|
cd apps/api && celery -A app.tasks.celery_app beat --loglevel=INFO
|
|
|
|
# Backend tests / lint / format
|
|
cd apps/api && pytest
|
|
cd apps/api && ruff check app tests
|
|
cd apps/api && black app tests
|
|
|
|
# Frontend tests / lint / format / typecheck
|
|
cd apps/web && npm test
|
|
cd apps/web && npm run lint
|
|
cd apps/web && npm run format
|
|
cd apps/web && npm run typecheck
|
|
|
|
# End-to-end test (requires `docker compose up -d` already running)
|
|
cd apps/web && npx playwright install --with-deps chromium # one-time
|
|
cd apps/web && npm run e2e
|
|
```
|
|
|
|
## Configuration
|
|
|
|
**A fresh clone ships with zero API keys, on purpose.** `.env` is gitignored and never committed; `.env.example` — the only thing that *is* committed — leaves every paid provider blank. There's no "secret" hidden somewhere in the repo history either; nothing paid has ever been checked in. Clone it, `docker compose up`, and the entire demo workflow (register/login, add a company, run a monitor, get a report, get an alert) works immediately against mock LLM/search providers and console-logged notifications — no signups, no billing, no keys, ever required just to try it.
|
|
|
|
Every paid/external provider defaults to a mock/console implementation this way. Automated tests always run against mocks too, and never call a paid API.
|
|
|
|
### Adding real provider keys
|
|
|
|
Once you have your own API keys, there are two independent places to put them, and they serve different purposes:
|
|
|
|
1. **`.env` (deployment-wide defaults)** — see [`.env.example`](.env.example) for the full list with comments. The key ones:
|
|
- `AUTH_MODE=local|jwt` — local single-user dev mode vs. real email/password accounts.
|
|
- `LLM_PROVIDER=mock|anthropic|ollama|gemini` — set `ANTHROPIC_API_KEY`/`ANTHROPIC_MODEL`, `OLLAMA_BASE_URL`/`OLLAMA_MODEL`, or `GEMINI_API_KEY`/`GEMINI_MODEL` to go live. Gemini has a genuine free tier — grab a key at [aistudio.google.com/apikey](https://aistudio.google.com/apikey) — so it's the cheapest provider to actually try against a real model.
|
|
- `SEARCH_PROVIDER=mock|brave` — set `BRAVE_SEARCH_API_KEY` to go live.
|
|
- `NINJAPEAR_API_KEY` / `USPTO_API_KEY` — optional company-enrichment and patent-search providers, both free-tier-friendly to start.
|
|
- `SMTP_HOST`/`RESEND_API_KEY` — no bundled local mail sink; point SMTP at a real relay or set `RESEND_API_KEY` to actually test email delivery (alerts and security email both use this).
|
|
- `NOTIFICATION_SMS_ENABLED=false` by default — set to `true` and provide `TWILIO_*` to enable SMS.
|
|
|
|
`LLM_PROVIDER`/`SEARCH_PROVIDER` are provider *selection* and only live here — after changing them, rebuild/restart `api`, `worker`, and `beat` to pick up the change.
|
|
|
|
2. **The app's own Settings page (no `.env` edit, no restart)** — once the app is running, sign in and go to Settings:
|
|
- **"Your API keys"** — every account can set its own Anthropic/Brave/NinjaPear/USPTO key, used for that account's own companies. Falls back to the `.env` value if you never set one.
|
|
- **"Server secrets"** (admin accounts only) — Cloudflare Turnstile site key/secret and the Resend API key, shared server-wide, stored encrypted in the database, and take effect immediately for every visitor. This is the intended way to configure these three on a real deployment rather than editing `.env` directly.
|
|
|
|
If `LLM_PROVIDER`/`SEARCH_PROVIDER` still show as `mock` in Settings → System configuration after you've set keys, that's the `.env`-level provider selection, not a missing key — it needs the `.env` edit + restart from option 1 above, not the Settings UI.
|
|
|
|
## Known limitations
|
|
|
|
See [`KNOWN_LIMITATIONS.md`](KNOWN_LIMITATIONS.md).
|