Compare commits

...
7 Commits
Author SHA1 Message Date
sakshamandClaude Sonnet 5 56b4a5404e Add pending account provisioning for demoing to not-yet-existing accounts
An admin can now queue up an email address in advance (POST/GET/DELETE
/system/pending-provisioning) with a source user and an optional admin
flag. The moment that email actually verifies a real account - not raw
registration, which proves nothing about ownership - it gets a deep copy
of the source user's per-user API keys and every company they own
(company profile, aliases, competitors, monitor config, sources, source
documents, monitoring runs, reports, snapshots, detected changes, and
enrichment - not just the company row), plus an email notification
destination for its own address linked to the copied companies.

The clone logic (_copy_row/_clone_company in provisioning_service.py) is
generic over every table it touches via column introspection, so it
doesn't need hand-maintained field lists per model.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-08-06 18:47:29 -04:00
sakshamandClaude Sonnet 5 18305b545c Backfill sparse report sections, add enrichment section refresh, and bootstrap first-admin
Reports: the LLM reliably used company_enrichment for prose fields but
inconsistently populated the parallel Finding-list/string-list fields from
the same evidence, even with progressively more explicit prompting. Add a
code-level backfill (products, recent developments, financial signals,
strategic initiatives, regulatory signals, risks/opportunities mirrored
from SWOT, unknowns, monitoring recommendations) that only ever fills in
what the model left empty, never overwrites what it produced.

Enrichment tab: reorder sections (Products/Recent updates before
Customers/Competitors) and add a per-section "Refresh" button that
re-fetches just one of NinjaPear's six independent per-company endpoints
when it came back empty - confirmed live that a data-coverage gap (e.g.
Amazon returning no products) is real provider behavior, not a bug.

Auth: the first account registered on a deployment with zero existing
admins is now auto-promoted to admin, closing the chicken-and-egg gap
where the only path to admin access was direct DB access. Self-heals if
the last admin ever deletes their account.

Also bumps nginx's proxy_read_timeout for api.ciagent.org to cover the
enrichment refresh's synchronous funding-endpoint call (up to 5 minutes
per NinjaPear's docs).

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-08-06 17:41:02 -04:00
sakshamandClaude Sonnet 5 1be3e53584 Clarify README: fresh clones ship with zero API keys, by design
Expands the Configuration section to explain why (nothing paid is
ever committed, .env is gitignored) and separates the two independent
places to add real keys once you have your own: .env for deployment-
wide provider selection/defaults, vs. the Settings page's "Your API
keys" (per-user) and admin-only "Server secrets" (Turnstile, Resend)
for a running deployment with no restart needed. Also links
DEPLOYMENT.md and the live ciagent.org/git.ciagent.org reference
deployment, and fixes the Celery queue list to include "enrichment".

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-08-06 08:14:14 -04:00
sakshamandClaude Sonnet 5 d706f226de Fix executable bit on scripts/*.sh, tracked as non-executable since commit
auto-deploy.sh only ever worked because someone chmod +x'd it directly
on the server after cloning, outside git - a fix that lived nowhere
git could see. A `git checkout --` to that file (recovering from an
unrelated direct edit) silently restored the tracked 644 mode,
breaking the deploy timer with "Permission denied" until caught via
journalctl. bootstrap-env.sh had the identical latent bug, just never
triggered since it's only ever run manually.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-08-06 07:13:42 -04:00
sakshamandClaude Sonnet 5 8ec7ca0881 Restart nginx on every deploy to avoid stale upstream IPs
nginx resolves the api/web service names to a container IP once, at
its own worker-process startup. docker compose up -d only recreates
containers whose image/config changed, so nginx keeps proxying to the
old, now-dead IP after a deploy rebuilds those containers - every
request 502s, which shows up in the browser as a misleading CORS
error since the bare 502 carries no Access-Control-Allow-Origin
header. Confirmed live: this caused a real multi-hour production
outage after the last two deploys.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-08-06 07:07:55 -04:00
sakshamandClaude Sonnet 5 4ee38b6241 Add DB viewer access logging, account deletion, and forced password change
Logs a distinct db_viewer_accessed event (not just the earlier
session_created "requested" event) when an admin's browser actually
completes the hand-off into Adminer. Adds a password-confirmed
account-deletion box to Settings, relying on the existing ON DELETE
CASCADE foreign keys to clean up everything the account owns. Adds an
admin-only "require password change" flag that get_current_user
enforces server-side (403 on everything except /auth/me,
/auth/change-password, /auth/logout) - meant for handing a demo
account to someone with a known sample password.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-08-05 23:41:21 -04:00
sakshamandClaude Sonnet 5 3d6fe56991 Add Settings -> Database viewer (Adminer) for local devs and any admin
Local dev gets an unauthenticated Adminer instance bound to loopback
only. In production, any account with is_admin=true can open it -
the app mints a short-lived token from a live admin session, which
Nginx's new db.ciagent.org block exchanges for a session cookie that
re-checks admin status on every request, instead of a shared static
password that wouldn't scale to multiple admins or revoke live.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-08-05 22:18:16 -04:00
54 changed files with 3713 additions and 75 deletions
+8
View File
@@ -22,6 +22,14 @@ NEXT_PUBLIC_API_URL=http://localhost:8000
# this if you run your own self-hosted git server and want the button to # this if you run your own self-hosted git server and want the button to
# point at your fork instead of upstream. # point at your fork instead of upstream.
NEXT_PUBLIC_GIT_REPO_URL= NEXT_PUBLIC_GIT_REPO_URL=
# "Open database viewer" link on the Settings page (local developers and
# server admins only). Access is gated by a short-lived token minted from a
# real admin login (see apps/api/app/api/v1/db_viewer.py), not a separate
# password. Blank by default - deliberately NOT defaulted the way
# NEXT_PUBLIC_GIT_REPO_URL is, since this points at your own deployment's
# private Adminer instance. Set to https://db.ciagent.org (or your own
# subdomain) once the db.ciagent.org Nginx block and DNS record exist.
NEXT_PUBLIC_DB_VIEWER_URL=
# --- Reverse proxy (only relevant once deployed behind Cloudflare/Nginx) ----- # --- Reverse proxy (only relevant once deployed behind Cloudflare/Nginx) -----
# Empty = trust the direct connection for client-IP resolution (correct for # Empty = trust the direct connection for client-IP resolution (correct for
+23 -6
View File
@@ -13,6 +13,7 @@ One Docker host runs everything via `docker-compose.prod.yml`: Postgres, Redis,
| `ciagent.org` | `web:3000` | Next.js frontend | | `ciagent.org` | `web:3000` | Next.js frontend |
| `api.ciagent.org` | `api:8000` | FastAPI backend | | `api.ciagent.org` | `api:8000` | FastAPI backend |
| `git.ciagent.org` | `gitea:3000` | Self-hosted git (public read, admin-only write) | | `git.ciagent.org` | `gitea:3000` | Self-hosted git (public read, admin-only write) |
| `db.ciagent.org` | `adminer:8080` | Database viewer (any admin account, no separate password - see §7) |
Cloudflare's proxy (orange-cloud DNS) hides the origin's real IP and absorbs generic bot/volumetric traffic; the app's own IP-based throttle/ban system (`app/services/ip_throttle_service.py`) handles the business-logic-specific abuse cases Cloudflare can't know about. Both need `TRUSTED_PROXY_IP_HEADER=CF-Connecting-IP` set correctly or IP-based logic breaks — see `KNOWN_LIMITATIONS.md`. Cloudflare's proxy (orange-cloud DNS) hides the origin's real IP and absorbs generic bot/volumetric traffic; the app's own IP-based throttle/ban system (`app/services/ip_throttle_service.py`) handles the business-logic-specific abuse cases Cloudflare can't know about. Both need `TRUSTED_PROXY_IP_HEADER=CF-Connecting-IP` set correctly or IP-based logic breaks — see `KNOWN_LIMITATIONS.md`.
@@ -24,8 +25,8 @@ Cloudflare's proxy (orange-cloud DNS) hides the origin's real IP and absorbs gen
## 2. Cloudflare DNS + Origin CA certificate ## 2. Cloudflare DNS + Origin CA certificate
1. Add three DNS **A records**, all proxied (orange cloud): `ciagent.org`, `api.ciagent.org`, `git.ciagent.org` → the server's public IP. 1. Add four DNS **A records**, all proxied (orange cloud): `ciagent.org`, `api.ciagent.org`, `git.ciagent.org`, `db.ciagent.org` → the server's public IP.
2. Cloudflare dashboard → SSL/TLS → **Origin Server** → Create Certificate. Cover `ciagent.org` and `*.ciagent.org` (one cert for all three subdomains), leave the default 15-year validity. Save the cert and private key. 2. Cloudflare dashboard → SSL/TLS → **Origin Server** → Create Certificate. Cover `ciagent.org` and `*.ciagent.org` (one cert for all four subdomains, including any added later), leave the default 15-year validity. Save the cert and private key.
3. On the server, create `/etc/ci-agent/certs/` (outside the repo, never committed) and place the two files there as `cloudflare-origin.pem` and `cloudflare-origin.key` — this is exactly what `docker-compose.prod.yml`'s `nginx` service mounts. 3. On the server, create `/etc/ci-agent/certs/` (outside the repo, never committed) and place the two files there as `cloudflare-origin.pem` and `cloudflare-origin.key` — this is exactly what `docker-compose.prod.yml`'s `nginx` service mounts.
4. Cloudflare dashboard → SSL/TLS → set the encryption mode to **Full (strict)**. Anything less either skips origin verification or falls back to plaintext HTTP to the origin. 4. Cloudflare dashboard → SSL/TLS → set the encryption mode to **Full (strict)**. Anything less either skips origin verification or falls back to plaintext HTTP to the origin.
@@ -59,6 +60,7 @@ FRONTEND_URL=https://ciagent.org
BACKEND_URL=https://api.ciagent.org BACKEND_URL=https://api.ciagent.org
NEXT_PUBLIC_API_URL=https://api.ciagent.org NEXT_PUBLIC_API_URL=https://api.ciagent.org
NEXT_PUBLIC_GIT_REPO_URL=https://git.ciagent.org/<your-admin-username>/ci-agent NEXT_PUBLIC_GIT_REPO_URL=https://git.ciagent.org/<your-admin-username>/ci-agent
NEXT_PUBLIC_DB_VIEWER_URL=https://db.ciagent.org
TRUSTED_PROXY_IP_HEADER=CF-Connecting-IP TRUSTED_PROXY_IP_HEADER=CF-Connecting-IP
RESEND_API_KEY=<real Resend key> # no admin-UI equivalent for this one RESEND_API_KEY=<real Resend key> # no admin-UI equivalent for this one
``` ```
@@ -97,7 +99,19 @@ Two ways to push, since Cloudflare's proxy only speaks HTTP(S) — raw SSH can't
Read-only clone/browse works for anyone, no account: `https://git.ciagent.org/<you>/ci-agent.git` or the web UI directly. Read-only clone/browse works for anyone, no account: `https://git.ciagent.org/<you>/ci-agent.git` or the web UI directly.
## 7. Post-boot: configure provider API keys via the Settings UI, not `.env` ## 7. Database viewer (Adminer at db.ciagent.org)
Unlike Gitea's admin account, there's no manual credential-handoff step for this one — any account with `is_admin=true` on the app itself can use it immediately, once `NEXT_PUBLIC_DB_VIEWER_URL` is set (§4) and the site is redeployed. Settings → Database → "Open database viewer" mints a short-lived token from the admin's real, live-checked login, which `db.ciagent.org`'s Nginx block (`infrastructure/nginx/nginx.conf`) exchanges for a signed session cookie via `apps/api/app/api/v1/db_viewer.py` — no separate password to generate, distribute, or rotate.
A couple of things worth knowing:
- Sessions last 60 minutes. Revoking someone's `is_admin` flag takes effect on their very next request through Nginx's `auth_request` check (it re-loads the user from the database each time), but an already-open Adminer tab isn't force-closed — it just stops being able to load anything new once that check runs again.
- Adminer's own Postgres login (username/password) is a second, independent layer past this gate — real DB credentials are still required to actually view or edit anything.
- An optional, commented-out IP-allowlist snippet is included in the `db.ciagent.org` Nginx block for admins with a static IP who want to require both the session *and* a matching source address — not enabled by default, since most admin connections don't have a stable IP to pin to.
## 8. Post-boot: configure provider API keys via the Settings UI, not `.env`
**Where does the admin account come from?** The very first account ever registered on the app (`is_admin` count is zero) is automatically promoted to admin - see `auth_service.register`. No manual DB access or setup script required, but it does mean the same race Gitea has in §5 applies here too: if the firewall is open and the site is reachable before you've registered your own account, a stranger who registers first becomes the admin instead of you. Register your own account immediately after the stack comes up, before opening the firewall/pointing DNS at it publicly. If the last admin's account is ever deleted, the next person to register becomes admin again - it never permanently locks the deployment out of admin access.
`scripts/bootstrap-env.sh` deliberately leaves these six blank. Sign in as the admin account and set them from the app itself: `scripts/bootstrap-env.sh` deliberately leaves these six blank. Sign in as the admin account and set them from the app itself:
@@ -106,7 +120,7 @@ Read-only clone/browse works for anyone, no account: `https://git.ciagent.org/<y
Also set `LLM_PROVIDER=anthropic` and `SEARCH_PROVIDER=brave` in `.env` (restart `api`/`worker`/`beat` after) once you've configured the corresponding keys above — provider *selection* is still a deployment-level `.env` setting, only the key *values* moved to the UI. Also set `LLM_PROVIDER=anthropic` and `SEARCH_PROVIDER=brave` in `.env` (restart `api`/`worker`/`beat` after) once you've configured the corresponding keys above — provider *selection* is still a deployment-level `.env` setting, only the key *values* moved to the UI.
## 8. Updating the deployment ## 9. Updating the deployment
Manually: Manually:
@@ -115,8 +129,11 @@ git pull
docker compose -f docker-compose.prod.yml build api worker beat web docker compose -f docker-compose.prod.yml build api worker beat web
docker compose -f docker-compose.prod.yml run --rm api alembic upgrade head docker compose -f docker-compose.prod.yml run --rm api alembic upgrade head
docker compose -f docker-compose.prod.yml up -d docker compose -f docker-compose.prod.yml up -d
docker compose -f docker-compose.prod.yml restart nginx
``` ```
**The `restart nginx` step is not optional.** `up -d` only recreates the containers whose image/config changed - nginx's own image/config doesn't change on an app deploy, so it keeps running with the container IPs it resolved at its *own* last startup. Once `api`/`web` get rebuilt with new IPs, nginx keeps proxying to the old, now-dead ones until something forces it to re-resolve - every request 502s (and shows up in the browser as a misleading CORS error, since a bare 502 from nginx carries no `Access-Control-Allow-Origin` header). Confirmed live: this caused a real multi-hour outage. `scripts/auto-deploy.sh` includes this step automatically.
**Or automatically**: `scripts/auto-deploy.sh` runs exactly that sequence, gated on "is `origin/master` ahead of `HEAD`" so it's a no-op most runs. Install it as a systemd timer (polls every 2 minutes — deliberately polling, not a Gitea webhook, so there's no extra exposed service, no Docker-socket-in-a-container, and no shared secret to manage): **Or automatically**: `scripts/auto-deploy.sh` runs exactly that sequence, gated on "is `origin/master` ahead of `HEAD`" so it's a no-op most runs. Install it as a systemd timer (polls every 2 minutes — deliberately polling, not a Gitea webhook, so there's no extra exposed service, no Docker-socket-in-a-container, and no shared secret to manage):
```bash ```bash
@@ -128,7 +145,7 @@ systemctl enable --now ci-agent-deploy.timer
Once running, pushing to `master` on Gitea is enough — the server picks it up within ~2 minutes, no manual SSH step needed. Check `journalctl -u ci-agent-deploy.service` to see deploy runs. Once running, pushing to `master` on Gitea is enough — the server picks it up within ~2 minutes, no manual SSH step needed. Check `journalctl -u ci-agent-deploy.service` to see deploy runs.
## 9. Backups ## 10. Backups
Nothing backs itself up by default. At minimum, a nightly cron job on the host: Nothing backs itself up by default. At minimum, a nightly cron job on the host:
@@ -142,7 +159,7 @@ docker run --rm -v ci-agent-prod_gitea-data:/data -v /backups:/backup alpine \
Copy `/backups` off-box (a Hetzner Storage Box via `rclone`, or any object storage) — local-only backups don't survive a lost disk. Retain a sane number of days and prune older ones. Copy `/backups` off-box (a Hetzner Storage Box via `rclone`, or any object storage) — local-only backups don't survive a lost disk. Retain a sane number of days and prune older ones.
## 10. Operational notes ## 11. Operational notes
- `beat` must stay exactly one instance, always — duplicate scheduled monitoring runs otherwise. Don't `--scale beat=2`. - `beat` must stay exactly one instance, always — duplicate scheduled monitoring runs otherwise. Don't `--scale beat=2`.
- Docker's log driver is capped per-service (`max-size: 10m`, `max-file: 3` in `docker-compose.prod.yml`) — nothing else bounds log growth on the host. - Docker's log driver is capped per-service (`max-size: 10m`, `max-file: 3` in `docker-compose.prod.yml`) — nothing else bounds log growth on the host.
+11 -3
View File
@@ -26,7 +26,7 @@ This file is updated as each phase lands. It exists so nothing is silently claim
- Access/refresh tokens are stored in `localStorage` on the frontend for simplicity (see `apps/web/lib/api-client.ts`). This is a reasonable tradeoff for a local-first MVP but is XSS-exposed compared to httpOnly cookies; a production multi-user deployment should move to cookie-based storage with CSRF protection before going live publicly. - Access/refresh tokens are stored in `localStorage` on the frontend for simplicity (see `apps/web/lib/api-client.ts`). This is a reasonable tradeoff for a local-first MVP but is XSS-exposed compared to httpOnly cookies; a production multi-user deployment should move to cookie-based storage with CSRF protection before going live publicly.
- There is no email verification, password reset, or account lockout after repeated failed logins yet (rate limiting mitigates brute force but doesn't lock the account). Password reset is called out in the spec as a "documented future capability" for the MVP. - There is no email verification, password reset, or account lockout after repeated failed logins yet (rate limiting mitigates brute force but doesn't lock the account). Password reset is called out in the spec as a "documented future capability" for the MVP.
- Admin role (`User.is_admin`) exists on the model and the local dev user is admin, but no admin-only endpoints exist yet to gate with `require_admin`. - **Admin role (`User.is_admin`) now gates several real endpoints** (`require_admin` - Server secrets, IP bans, Database viewer, system logs, unban-request review) - this note originally said none existed yet, long since outdated. **The first account ever registered on a deployment with zero existing admins is now automatically promoted to admin** (`auth_service.register`, checked via `UserRepository.count_admins`) - otherwise the only path to admin access was direct DB manipulation, a real chicken-and-egg problem for anyone self-hosting from a clean clone (see DEPLOYMENT.md §8). Checked by admin *count*, not total user count, so it also self-heals if the last admin ever deletes their own account - the next registration becomes admin again rather than leaving the deployment permanently admin-less. A benign race is possible if two people register in the same instant on a brand-new, zero-admin deployment (both could become admin) - acceptable since it only ever matters once, before any real traffic exists, and DEPLOYMENT.md tells operators to register immediately after bringing the stack up, before opening the firewall.
## Collection pipeline (Phase 4) ## Collection pipeline (Phase 4)
@@ -57,6 +57,7 @@ This file is updated as each phase lands. It exists so nothing is silently claim
- **Tasks A (relevance), B (extraction), and C (synthesis) are implemented and tested but not yet wired into the collection/report pipeline** - Task D (report generation) currently consumes raw `SourceDocument`/`DetectedChange` rows directly rather than pre-filtering through Task A or pre-extracting via Task B/C. Wiring them in would improve report quality (e.g. filtering irrelevant documents before they reach the report prompt) but the report already works correctly without it, since `report_service.py`'s evidence-gathering only pulls documents/changes already scoped to the company. - **Tasks A (relevance), B (extraction), and C (synthesis) are implemented and tested but not yet wired into the collection/report pipeline** - Task D (report generation) currently consumes raw `SourceDocument`/`DetectedChange` rows directly rather than pre-filtering through Task A or pre-extracting via Task B/C. Wiring them in would improve report quality (e.g. filtering irrelevant documents before they reach the report prompt) but the report already works correctly without it, since `report_service.py`'s evidence-gathering only pulls documents/changes already scoped to the company.
- **A report is regenerated only on baseline (first evidence) or when a run detects an actual change** - not on every scheduled run, to keep LLM cost proportional to real activity rather than to schedule cadence. See `app/tasks/collection.py`. - **A report is regenerated only on baseline (first evidence) or when a run detects an actual change** - not on every scheduled run, to keep LLM cost proportional to real activity rather than to schedule cadence. See `app/tasks/collection.py`.
- **No token/cost usage tracking or per-user LLM budget yet** - `LLM_MAX_TOKENS_PER_REQUEST` and `LLM_MAX_RETRIES` bound a single call, but there's no aggregate usage dashboard (spec section 30's "Display usage statistics in settings" is Phase 10/not-yet-done). - **No token/cost usage tracking or per-user LLM budget yet** - `LLM_MAX_TOKENS_PER_REQUEST` and `LLM_MAX_RETRIES` bound a single call, but there's no aggregate usage dashboard (spec section 30's "Display usage statistics in settings" is Phase 10/not-yet-done).
- **Report list/string fields are backfilled from `company_enrichment` when the LLM leaves them empty** - confirmed live across multiple real Anthropic calls (even with an explicit worked example in the prompt) that the model reliably uses enrichment data for prose fields (`company_overview`, `market_positioning`) but is inconsistent about mirroring the same data into list-shaped fields (`products_and_services`, `financial_signals`, `strategic_initiatives`, etc.). `report_service.py`'s `_backfill_from_enrichment`/`_mirror_swot_into_flat_lists`/`_backfill_reflective_sections` fill these in mechanically from `company_enrichment.products`/`recent_updates`/`funding`/`specialties`, from the model's own SWOT output, and from the actual presence/absence of evidence - never overwriting content the model did produce, and never inventing anything not already present in real evidence. **`key_inferred_projects` is the one field left with no such fallback** - it asks for specific, speculative in-progress projects, which has no honest mechanical source (unlike a specialty list or a funding round, there's no real "raw project" data to transcribe), so it stays genuinely empty unless the model or actual source documents/detected changes surface something.
## Notifications (Phase 8) ## Notifications (Phase 8)
@@ -132,7 +133,7 @@ This file is updated as each phase lands. It exists so nothing is silently claim
- **The "Similar People" endpoint is not wired in at all** - it's a role/company-anchored prospecting tool ("find people like X at competitor Y") with no natural trigger at onboarding time, since no specific role is ever selected. Would need its own UI (e.g. "find people like this one" on a leadership-team entry) to make sense, not an automatic onboarding call. - **The "Similar People" endpoint is not wired in at all** - it's a role/company-anchored prospecting tool ("find people like X at competitor Y") with no natural trigger at onboarding time, since no specific role is ever selected. Would need its own UI (e.g. "find people like this one" on a leadership-team entry) to make sense, not an automatic onboarding call.
- **NinjaPear's suggested competitors are never merged into `Company.competitors`** (the user-typed/reviewed list that drives Phase 12's competitor cross-linking) - shown only in the Enrichment tab, with their stated reasons, kept deliberately separate so a background API call can never silently change what the user explicitly reviewed. A "add as competitor" action would be a reasonable follow-up if this becomes a papercut. - **NinjaPear's suggested competitors are never merged into `Company.competitors`** (the user-typed/reviewed list that drives Phase 12's competitor cross-linking) - shown only in the Enrichment tab, with their stated reasons, kept deliberately separate so a background API call can never silently change what the user explicitly reviewed. A "add as competitor" action would be a reasonable follow-up if this becomes a papercut.
- **Person-level lookups (work email, profile) are capped at `NINJAPEAR_MAX_LEADERSHIP_LOOKUPS` (default 5) per company**, applied to however many leadership members NinjaPear's Company Details call happens to return, in whatever order it returns them - there's no ranking by seniority/relevance before the cap is applied, so for a company with a large leadership team, which specific people get resolved is effectively arbitrary. - **Person-level lookups (work email, profile) are capped at `NINJAPEAR_MAX_LEADERSHIP_LOOKUPS` (default 5) per company**, applied to however many leadership members NinjaPear's Company Details call happens to return, in whatever order it returns them - there's no ranking by seniority/relevance before the cap is applied, so for a company with a large leadership team, which specific people get resolved is effectively arbitrary.
- **No re-enrichment mechanism** - by design (see Phase 15's Context in `TASKS.md`, onboarding-only was the explicit scoping decision to bound cost), but it does mean `CompanyEnrichment.data` can go stale indefinitely (a funding round happens, a leadership change occurs) with no way to refresh it short of manually deleting the row and re-triggering via the API/DB directly - there's no "re-enrich this company" button anywhere in the UI. - **No automatic re-enrichment** - by design (see Phase 15's Context in `TASKS.md`, onboarding-only was the explicit scoping decision to bound cost), so `CompanyEnrichment.data` can go stale indefinitely (a funding round happens, a leadership change occurs) with no scheduled refresh. **A manual, per-section refresh does exist** (`POST /companies/{id}/enrichment/sections/{section}/refresh`, `enrichment_service.refresh_section`): each of NinjaPear's six per-company endpoints (details, funding, updates, competitors, products, customers) maps to its own UI section, and a "Refresh" button appears next to a section's empty-state message ("None found.") so a genuine provider-side data gap (confirmed live: NinjaPear's real API returns `products: []` for Amazon specifically, with no error - not a bug, just no product data on their end for that company) can be manually retried without re-running the whole enrichment or burning credits on the sections that already succeeded. Runs synchronously in the request (not queued via Celery like onboarding enrichment), since it's a single provider call rather than the full six-plus-leadership fan-out - nginx's `proxy_read_timeout` on `api.ciagent.org` is set to 320s to cover `funding`'s documented up-to-5-minute worst case.
- **`GET /system/status`'s live credit-balance check adds one outbound HTTPS call (to NinjaPear) to every load of that endpoint** whenever a key is configured - guarded so a failure there can never break the rest of system status (returns `null` for the balance, logs a warning), but it does mean the Settings page's load time now has a dependency on `nubela.co`'s availability/latency for that one field, not just the app's own DB/Redis health. - **`GET /system/status`'s live credit-balance check adds one outbound HTTPS call (to NinjaPear) to every load of that endpoint** whenever a key is configured - guarded so a failure there can never break the rest of system status (returns `null` for the balance, logs a warning), but it does mean the Settings page's load time now has a dependency on `nubela.co`'s availability/latency for that one field, not just the app's own DB/Redis health.
## Live logging, API-key visibility, and enrichment-tab formatting (Phase 17) ## Live logging, API-key visibility, and enrichment-tab formatting (Phase 17)
@@ -169,7 +170,7 @@ This file is updated as each phase lands. It exists so nothing is silently claim
- **The old admin-only, localhost-only "API keys" Settings box (`/system/api-keys`, Phase 17) has been removed entirely**, replaced by two independent mechanisms with a deliberately different visibility model each: per-user API keys (Anthropic/Brave/NinjaPear/USPTO - every user manages their own, no admin/localhost gate at all, since each user only ever sees their own value) and a new admin-only (but **not** localhost-gated) "Server secrets" box for values that are genuinely global rather than per-user - today just the Cloudflare Turnstile site key and secret. `SystemSecret` (`app/models/system_secret.py`) is a true singleton-per-key table (one row per `SystemSecretKey`, not scoped to any user), encrypted at rest the same way as `UserApiKey`, with the identical fallback-to-`.env`-when-unset pattern via `system_secret_service.get_effective_settings`. - **The old admin-only, localhost-only "API keys" Settings box (`/system/api-keys`, Phase 17) has been removed entirely**, replaced by two independent mechanisms with a deliberately different visibility model each: per-user API keys (Anthropic/Brave/NinjaPear/USPTO - every user manages their own, no admin/localhost gate at all, since each user only ever sees their own value) and a new admin-only (but **not** localhost-gated) "Server secrets" box for values that are genuinely global rather than per-user - today just the Cloudflare Turnstile site key and secret. `SystemSecret` (`app/models/system_secret.py`) is a true singleton-per-key table (one row per `SystemSecretKey`, not scoped to any user), encrypted at rest the same way as `UserApiKey`, with the identical fallback-to-`.env`-when-unset pattern via `system_secret_service.get_effective_settings`.
- **The Turnstile site key is no longer delivered to the frontend via a `NEXT_PUBLIC_*` build-time env var.** It used to be baked into the Next.js bundle at container-build time (`NEXT_PUBLIC_TURNSTILE_SITE_KEY`), which meant an admin-updated value could never take effect without a full frontend rebuild - defeating the point of making it admin-editable. It's now served live by the already-public, unauthenticated `GET /system/status` endpoint (`turnstile_site_key`, resolved through the same effective-settings fallback), and `TurnstileWidget` takes it as a prop instead of reading `process.env` directly. This does mean the site key round-trips through one more network hop (an extra field on a call the login/register pages already make) rather than being inlined - a negligible cost for a value that isn't itself secret. - **The Turnstile site key is no longer delivered to the frontend via a `NEXT_PUBLIC_*` build-time env var.** It used to be baked into the Next.js bundle at container-build time (`NEXT_PUBLIC_TURNSTILE_SITE_KEY`), which meant an admin-updated value could never take effect without a full frontend rebuild - defeating the point of making it admin-editable. It's now served live by the already-public, unauthenticated `GET /system/status` endpoint (`turnstile_site_key`, resolved through the same effective-settings fallback), and `TurnstileWidget` takes it as a prop instead of reading `process.env` directly. This does mean the site key round-trips through one more network hop (an extra field on a call the login/register pages already make) rather than being inlined - a negligible cost for a value that isn't itself secret.
- **`system_secrets` rows are true global singletons with no natural per-test isolation** (unlike `user_api_keys`, which is naturally isolated by a randomized per-test `user_id`) - `test_system_secrets.py` has to explicitly clear the table before and after every test via an autouse fixture to avoid cross-test pollution within a full-suite run. Worth remembering if this pattern is extended to more server-wide keys later: any new key added to `SystemSecretKey` inherits the same test-isolation requirement. - **`system_secrets` rows are true global singletons with no natural per-test isolation** (unlike `user_api_keys`, which is naturally isolated by a randomized per-test `user_id`) - `test_system_secrets.py` has to explicitly clear the table before and after every test via an autouse fixture to avoid cross-test pollution within a full-suite run. Worth remembering if this pattern is extended to more server-wide keys later: any new key added to `SystemSecretKey` inherits the same test-isolation requirement.
- **The real `.env` `TURNSTILE_SITE_KEY`/`TURNSTILE_SECRET` values were copied into this DB-backed storage** (both now `configured: true` via the admin Settings box) rather than removed from `.env` outright - the user plans to purge the `.env` values manually once satisfied the DB-backed path works, so both currently agree and nothing changes behaviorally either way. `NEXT_PUBLIC_TURNSTILE_SITE_KEY` specifically *was* removed from `.env`/`.env.example`/`docker-compose.yml`, since nothing reads it anymore (see above) - that one wasn't a "purge later" judgment call, it was genuinely dead as soon as the frontend stopped reading it. - **The real `.env` `TURNSTILE_SITE_KEY`/`TURNSTILE_SECRET`/`RESEND_API_KEY` values on the production deployment have since been moved into this DB-backed storage via the admin API and removed from `.env` outright** (all three now `configured: true`, set through `PUT /system/secrets/{key}` using the deployment's own admin account rather than hand-edited into `.env`) - the earlier state of this note (values copied into the DB but also left in `.env` "to purge later") has been resolved; `.env` is no longer a source of truth for these three at all on that deployment. `NEXT_PUBLIC_TURNSTILE_SITE_KEY` specifically *was already* removed from `.env`/`.env.example`/`docker-compose.yml` earlier, since nothing reads it anymore (see above).
## Account activity logging expansion: local-dev sign-ins, secret/key updates, known IPs ## Account activity logging expansion: local-dev sign-ins, secret/key updates, known IPs
@@ -178,4 +179,11 @@ This file is updated as each phase lands. It exists so nothing is silently claim
- **New `user_known_ips` table captures every distinct IP an account has signed in from** (`app/models/user_known_ip.py`), one row per `(user_id, ip_address)` pair with `first_seen_at`/`last_seen_at`, updated on every recorded sign-in (both real login and the local-dev bypass, gated by the same cooldown above for the latter). This is pure data capture for now - **nothing currently reads this table or surfaces it anywhere in the UI**; it exists as the foundation for a possible future "new device/location" security feature, per explicit request. `UserKnownIpRepository.record_login`'s return value (whether the IP was new) is already threaded through but currently unused by any caller. - **New `user_known_ips` table captures every distinct IP an account has signed in from** (`app/models/user_known_ip.py`), one row per `(user_id, ip_address)` pair with `first_seen_at`/`last_seen_at`, updated on every recorded sign-in (both real login and the local-dev bypass, gated by the same cooldown above for the latter). This is pure data capture for now - **nothing currently reads this table or surfaces it anywhere in the UI**; it exists as the foundation for a possible future "new device/location" security feature, per explicit request. `UserKnownIpRepository.record_login`'s return value (whether the IP was new) is already threaded through but currently unused by any caller.
- **IPs were already being recorded per-login inside `user_security_events`** (every `login_success`/`login_failed` row has always carried `ip_address`) - `user_known_ips` doesn't replace that, it's a deliberately separate, deduplicated view: the security-events log is an append-only history of every attempt, while `user_known_ips` answers "what's the current set of IPs this account has ever used" without needing to scan and dedupe the (much larger, unbounded-growth - see the Phase 19 addendum above) events table. - **IPs were already being recorded per-login inside `user_security_events`** (every `login_success`/`login_failed` row has always carried `ip_address`) - `user_known_ips` doesn't replace that, it's a deliberately separate, deduplicated view: the security-events log is an append-only history of every attempt, while `user_known_ips` answers "what's the current set of IPs this account has ever used" without needing to scan and dedupe the (much larger, unbounded-growth - see the Phase 19 addendum above) events table.
## Settings -> Database viewer (Adminer, local-dev/admin only)
- **This feature deliberately does not reuse the app's own JWT/admin session to gate access, because it can't.** The frontend's access token lives only in `window.localStorage` (`apps/web/lib/api-client.ts`), never a cookie, and is attached only as a JS-constructed `Authorization` header on this app's own `fetch()` calls - a plain browser navigation to a different subdomain (`db.ciagent.org`) carries none of that. Instead, `apps/api/app/api/v1/db_viewer.py` mints a short-lived bootstrap token from a live, `require_admin`-gated session, which Nginx's `db.ciagent.org` block exchanges for an independent, cookie-based session scoped to that subdomain only. Both token types reuse the same `JWT_SECRET`/`TokenType` machinery as real login tokens (`apps/api/app/core/security.py`) rather than introducing a second signing secret.
- **The `verify` endpoint re-loads the user and re-checks `is_admin` from the database on every request** (not just at token-mint time), so revoking someone's admin flag takes effect on their very next request through Nginx's `auth_request` - but a DB_VIEWER_SESSION cookie already issued is otherwise valid for its full 60-minute lifetime; there's no server-side session revocation list, only the live `is_admin` check and natural expiry.
- **Local dev's Adminer (`docker-compose.yml`, port `127.0.0.1:8081`) has no authentication of its own at all** - it relies entirely on the port being bound to loopback only, matching this app's existing "loopback is inherently trusted" philosophy elsewhere (e.g. the local-dev auth bypass itself). Anyone who can reach `localhost:8081` on that machine - including another local user account on a shared machine - has full Postgres access with no further gate.
- **Adminer itself has no read-only mode** - the feature was explicitly requested as "view and edit," so there's no additional restriction at the Adminer-config layer beyond Nginx's session gate (production) or loopback binding (local dev). The real Postgres username/password, required by Adminer's own login form, is the only remaining layer once past those.
Further limitations are appended per-phase below. Further limitations are appended per-phase below.
+25 -9
View File
@@ -2,7 +2,9 @@
CI Agent monitors companies you choose, collects publicly available information from multiple sources on a schedule, analyzes it with an LLM into an evidence-linked report, detects meaningful changes between runs, and alerts you by email (and optionally SMS) with a severity and confidence score. CI Agent monitors companies you choose, collects publicly available information from multiple sources on a schedule, analyzes it with an LLM into an evidence-linked report, detects meaningful changes between runs, and alerts you by email (and optionally SMS) with a severity and confidence score.
See [`PLAN.md`](PLAN.md) for the build strategy, [`ARCHITECTURE.md`](ARCHITECTURE.md) for system design, [`SECURITY.md`](SECURITY.md) for the threat model, [`TASKS.md`](TASKS.md) for the live implementation checklist, [`KNOWN_LIMITATIONS.md`](KNOWN_LIMITATIONS.md) for what's stubbed vs. fully live, and [`docs/FIREBASE_MIGRATION.md`](docs/FIREBASE_MIGRATION.md) for a (skeptical) look at what moving to Firebase would take. See [`PLAN.md`](PLAN.md) for the build strategy, [`ARCHITECTURE.md`](ARCHITECTURE.md) for system design, [`SECURITY.md`](SECURITY.md) for the threat model, [`TASKS.md`](TASKS.md) for the live implementation checklist, [`KNOWN_LIMITATIONS.md`](KNOWN_LIMITATIONS.md) for what's stubbed vs. fully live, [`DEPLOYMENT.md`](DEPLOYMENT.md) for running this in production behind Cloudflare/Nginx, and [`docs/FIREBASE_MIGRATION.md`](docs/FIREBASE_MIGRATION.md) for a (skeptical) look at what moving to Firebase would take.
A live reference deployment runs at [ciagent.org](https://ciagent.org), with its own code hosted on a self-hosted Gitea instance at [git.ciagent.org](https://git.ciagent.org) (public read, admin-only write) — click "Git Repository" on the landing page.
## Quick start (Docker) ## Quick start (Docker)
@@ -57,7 +59,7 @@ cd apps/api && alembic upgrade head
cd apps/api && alembic revision --autogenerate -m "description" cd apps/api && alembic revision --autogenerate -m "description"
# Run a Celery worker + beat (only needed outside Docker) # Run a Celery worker + beat (only needed outside Docker)
cd apps/api && celery -A app.tasks.celery_app worker --loglevel=INFO -Q default,collection,analysis,notifications,maintenance cd apps/api && celery -A app.tasks.celery_app worker --loglevel=INFO -Q default,collection,analysis,notifications,maintenance,enrichment
cd apps/api && celery -A app.tasks.celery_app beat --loglevel=INFO cd apps/api && celery -A app.tasks.celery_app beat --loglevel=INFO
# Backend tests / lint / format # Backend tests / lint / format
@@ -78,15 +80,29 @@ cd apps/web && npm run e2e
## Configuration ## Configuration
All configuration is via environment variables — see [`.env.example`](.env.example) for the full list with comments. Highlights: **A fresh clone ships with zero API keys, on purpose.** `.env` is gitignored and never committed; `.env.example` — the only thing that *is* committed — leaves every paid provider blank. There's no "secret" hidden somewhere in the repo history either; nothing paid has ever been checked in. Clone it, `docker compose up`, and the entire demo workflow (register/login, add a company, run a monitor, get a report, get an alert) works immediately against mock LLM/search providers and console-logged notifications — no signups, no billing, no keys, ever required just to try it.
- `AUTH_MODE=local|jwt` — local single-user dev mode vs. real email/password accounts. Every paid/external provider defaults to a mock/console implementation this way. Automated tests always run against mocks too, and never call a paid API.
- `LLM_PROVIDER=mock|anthropic|ollama|gemini` — set `ANTHROPIC_API_KEY`/`ANTHROPIC_MODEL`, `OLLAMA_BASE_URL`/`OLLAMA_MODEL`, or `GEMINI_API_KEY`/`GEMINI_MODEL` to go live. Gemini has a genuine free tier — grab a key at [aistudio.google.com/apikey](https://aistudio.google.com/apikey) — so it's the cheapest provider to actually try against a real model.
- `SEARCH_PROVIDER=mock|brave` — set `BRAVE_SEARCH_API_KEY` to go live.
- `SMTP_HOST`/`RESEND_API_KEY` — no bundled local mail sink; point SMTP at a real relay or set `RESEND_API_KEY` to actually test email delivery (alerts and security email both use this).
- `NOTIFICATION_SMS_ENABLED=false` by default — set to `true` and provide `TWILIO_*` to enable SMS.
Every paid/external provider defaults to a mock/console implementation. Automated tests always run against mocks and never call a paid API. ### Adding real provider keys
Once you have your own API keys, there are two independent places to put them, and they serve different purposes:
1. **`.env` (deployment-wide defaults)** — see [`.env.example`](.env.example) for the full list with comments. The key ones:
- `AUTH_MODE=local|jwt` — local single-user dev mode vs. real email/password accounts.
- `LLM_PROVIDER=mock|anthropic|ollama|gemini` — set `ANTHROPIC_API_KEY`/`ANTHROPIC_MODEL`, `OLLAMA_BASE_URL`/`OLLAMA_MODEL`, or `GEMINI_API_KEY`/`GEMINI_MODEL` to go live. Gemini has a genuine free tier — grab a key at [aistudio.google.com/apikey](https://aistudio.google.com/apikey) — so it's the cheapest provider to actually try against a real model.
- `SEARCH_PROVIDER=mock|brave` — set `BRAVE_SEARCH_API_KEY` to go live.
- `NINJAPEAR_API_KEY` / `USPTO_API_KEY` — optional company-enrichment and patent-search providers, both free-tier-friendly to start.
- `SMTP_HOST`/`RESEND_API_KEY` — no bundled local mail sink; point SMTP at a real relay or set `RESEND_API_KEY` to actually test email delivery (alerts and security email both use this).
- `NOTIFICATION_SMS_ENABLED=false` by default — set to `true` and provide `TWILIO_*` to enable SMS.
`LLM_PROVIDER`/`SEARCH_PROVIDER` are provider *selection* and only live here — after changing them, rebuild/restart `api`, `worker`, and `beat` to pick up the change.
2. **The app's own Settings page (no `.env` edit, no restart)** — once the app is running, sign in and go to Settings:
- **"Your API keys"** — every account can set its own Anthropic/Brave/NinjaPear/USPTO key, used for that account's own companies. Falls back to the `.env` value if you never set one.
- **"Server secrets"** (admin accounts only) — Cloudflare Turnstile site key/secret and the Resend API key, shared server-wide, stored encrypted in the database, and take effect immediately for every visitor. This is the intended way to configure these three on a real deployment rather than editing `.env` directly.
If `LLM_PROVIDER`/`SEARCH_PROVIDER` still show as `mock` in Settings → System configuration after you've set keys, that's the `.env`-level provider selection, not a missing key — it needs the `.env` edit + restart from option 1 above, not the Settings UI.
## Known limitations ## Known limitations
+19 -1
View File
@@ -15,13 +15,14 @@ from app.core.security import get_client_ip
from app.db.session import get_db from app.db.session import get_db
from app.models.user import User from app.models.user import User
from app.repositories.unban_request_repository import UnbanRequestRepository from app.repositories.unban_request_repository import UnbanRequestRepository
from app.schemas.auth import RequirePasswordChangeRequest
from app.schemas.unban import ( from app.schemas.unban import (
BanIpRequest, BanIpRequest,
IpBanResponse, IpBanResponse,
UnbanRequestPayload, UnbanRequestPayload,
UnbanRequestResponse, UnbanRequestResponse,
) )
from app.services import unban_service from app.services import auth_service, unban_service
router = APIRouter(tags=["admin"]) router = APIRouter(tags=["admin"])
@@ -89,3 +90,20 @@ async def reject_unban_request(
_admin: User = Depends(require_admin), _admin: User = Depends(require_admin),
) -> None: ) -> None:
await unban_service.reject_unban_request(db, request_id) await unban_service.reject_unban_request(db, request_id)
@router.post("/admin/users/require-password-change", status_code=status.HTTP_204_NO_CONTENT)
async def require_password_change(
request: Request,
payload: RequirePasswordChangeRequest,
settings: Settings = Depends(get_settings),
db: AsyncSession = Depends(get_db),
_admin: User = Depends(require_admin),
) -> None:
"""Flags an account (by email) so its next request is blocked
everywhere except /auth/me, /auth/change-password, and /auth/logout
until they set a new password - see app.auth.dependencies.get_current_user
and app.services.auth_service.require_password_change. Meant for handing
a demo account to someone with a known sample password."""
client_ip = get_client_ip(request, settings)
await auth_service.require_password_change(db, client_ip, payload.email)
+23
View File
@@ -14,7 +14,9 @@ from app.core.security import get_client_ip, is_localhost
from app.db.session import get_db from app.db.session import get_db
from app.models.user import LOCAL_DEV_USER_ID, User from app.models.user import LOCAL_DEV_USER_ID, User
from app.schemas.auth import ( from app.schemas.auth import (
ChangePasswordRequest,
ConfirmPasswordResetRequest, ConfirmPasswordResetRequest,
DeleteAccountRequest,
LoginRequest, LoginRequest,
LogoutRequest, LogoutRequest,
RefreshRequest, RefreshRequest,
@@ -166,3 +168,24 @@ async def me(user: User = Depends(get_current_user)) -> MeResponse:
effective_auth_mode = "local" if user.id == LOCAL_DEV_USER_ID else "jwt" effective_auth_mode = "local" if user.id == LOCAL_DEV_USER_ID else "jwt"
base = UserResponse.model_validate(user).model_dump() base = UserResponse.model_validate(user).model_dump()
return MeResponse(**base, auth_mode=effective_auth_mode) return MeResponse(**base, auth_mode=effective_auth_mode)
@router.delete("/me", status_code=status.HTTP_204_NO_CONTENT)
async def delete_account(
payload: DeleteAccountRequest,
db: AsyncSession = Depends(get_db),
user: User = Depends(get_current_user),
) -> None:
await auth_service.delete_account(db, user, payload.password)
@router.post("/change-password", response_model=TokenResponse)
async def change_password(
request: Request,
payload: ChangePasswordRequest,
settings: Settings = Depends(get_settings),
db: AsyncSession = Depends(get_db),
user: User = Depends(get_current_user),
) -> TokenResponse:
client_ip = get_client_ip(request, settings)
return await auth_service.change_password(db, settings, client_ip, user, payload)
+32 -1
View File
@@ -3,6 +3,7 @@
from __future__ import annotations from __future__ import annotations
import uuid import uuid
from typing import Literal
from fastapi import APIRouter, Depends, Request, status from fastapi import APIRouter, Depends, Request, status
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
@@ -13,6 +14,7 @@ from app.core.config import Settings, get_settings
from app.core.errors import NotFoundError from app.core.errors import NotFoundError
from app.core.rate_limit import limiter from app.core.rate_limit import limiter
from app.db.session import get_db from app.db.session import get_db
from app.enrichment.factory import get_enrichment_provider
from app.models.user import User from app.models.user import User
from app.schemas.company import ( from app.schemas.company import (
CompanyCreate, CompanyCreate,
@@ -23,7 +25,14 @@ from app.schemas.company import (
) )
from app.schemas.discovery import DiscoverCompanyRequest, DiscoveredCompanyProfile from app.schemas.discovery import DiscoverCompanyRequest, DiscoveredCompanyProfile
from app.search.factory import get_search_provider from app.search.factory import get_search_provider
from app.services import company_service, discovery_service, user_api_key_service from app.services import (
company_service,
discovery_service,
enrichment_service,
user_api_key_service,
)
EnrichmentSection = Literal["details", "funding", "updates", "competitors", "products", "customers"]
router = APIRouter(prefix="/companies", tags=["companies"]) router = APIRouter(prefix="/companies", tags=["companies"])
@@ -129,6 +138,28 @@ async def resume_company(
return CompanyResponse.from_company(company) return CompanyResponse.from_company(company)
@router.post("/{company_id}/enrichment/sections/{section}/refresh", response_model=CompanyResponse)
@limiter.limit("10/minute")
async def refresh_enrichment_section(
request: Request,
company_id: uuid.UUID,
section: EnrichmentSection,
user: User = Depends(get_current_user),
settings: Settings = Depends(get_settings),
db: AsyncSession = Depends(get_db),
) -> CompanyResponse:
"""Re-fetches one enrichment section (e.g. `products`) that came back
empty or failed, without touching the other sections - see
enrichment_service.refresh_section. Rate-limited since each call spends
real NinjaPear credits."""
company = await company_service.get_company(db, user.id, company_id)
settings = await user_api_key_service.get_effective_settings(db, user.id, settings)
provider = get_enrichment_provider(settings)
await enrichment_service.refresh_section(db, settings, provider, company, section)
refreshed = await company_service.get_company(db, user.id, company_id)
return CompanyResponse.from_company(refreshed)
@router.get("/{company_id}/monitor", response_model=MonitorConfigurationResponse) @router.get("/{company_id}/monitor", response_model=MonitorConfigurationResponse)
async def get_monitor_configuration( async def get_monitor_configuration(
company_id: uuid.UUID, company_id: uuid.UUID,
+146
View File
@@ -0,0 +1,146 @@
"""Settings -> Database viewer: lets any admin open a web-based Postgres
client (Adminer, at db.<domain>) without a separate, shared credential.
The app's own JWT lives only in the browser's localStorage
(apps/web/lib/api-client.ts) - never a cookie, never sent on a plain
cross-subdomain navigation - so it can't gate a link to a different
subdomain the way an in-app API call is gated. Instead: an admin mints a
short-lived DB_VIEWER_BOOTSTRAP token from their real (live-checked) admin
session; Nginx's db.<domain> /_auth route exchanges that, one-time, for a
longer-lived DB_VIEWER_SESSION cookie scoped to that subdomain; Nginx's
`auth_request` then re-validates that cookie - including a fresh `is_admin`
check against the database - on every subsequent request. See
infrastructure/nginx/nginx.conf's db.<domain> server block and
DEPLOYMENT.md.
"""
from __future__ import annotations
from fastapi import APIRouter, Depends, HTTPException, Query, Request, Response, status
from fastapi.responses import RedirectResponse
from pydantic import BaseModel
from sqlalchemy.ext.asyncio import AsyncSession
from app.auth.dependencies import require_admin
from app.core.config import Settings, get_settings
from app.core.security import (
InvalidTokenError,
TokenType,
create_db_viewer_bootstrap_token,
create_db_viewer_session_token,
decode_token,
get_client_ip,
)
from app.db.session import get_db
from app.models.enums import SecurityEventType
from app.models.user import User
from app.repositories.user_repository import UserRepository
from app.repositories.user_security_event_repository import UserSecurityEventRepository
router = APIRouter(prefix="/db-viewer", tags=["db-viewer"])
SESSION_COOKIE_NAME = "db_viewer_session"
class DbViewerSessionResponse(BaseModel):
token: str
@router.post("/session", response_model=DbViewerSessionResponse)
async def create_db_viewer_session(
request: Request,
settings: Settings = Depends(get_settings),
db: AsyncSession = Depends(get_db),
admin: User = Depends(require_admin),
) -> DbViewerSessionResponse:
"""Mints a 2-minute bootstrap token the frontend embeds in a link to
https://<db-viewer-host>/_auth?token=... - Nginx proxies that path
straight to `bootstrap` below, which exchanges it for a session cookie."""
token = create_db_viewer_bootstrap_token(admin.id, settings)
await UserSecurityEventRepository(db).create(
user_id=admin.id,
event_type=SecurityEventType.DB_VIEWER_SESSION_CREATED,
ip_address=get_client_ip(request, settings),
)
await db.commit()
return DbViewerSessionResponse(token=token)
@router.get("/bootstrap")
async def bootstrap_db_viewer_session(
request: Request,
token: str = Query(...),
settings: Settings = Depends(get_settings),
db: AsyncSession = Depends(get_db),
) -> RedirectResponse:
"""Reached only via Nginx's db.<domain> /_auth location - never called
directly by the frontend. No `require_admin` dependency: the bootstrap
token itself, freshly minted by `create_db_viewer_session` above, is the
credential here."""
try:
decoded = decode_token(token, settings, TokenType.DB_VIEWER_BOOTSTRAP)
except InvalidTokenError as exc:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN, detail="Invalid or expired token"
) from exc
# Logged here, not at /session mint time, because this is the point the
# admin's browser actually completes the hand-off into Adminer - minting
# a token only proves they clicked the button, not that they got in.
await UserSecurityEventRepository(db).create(
user_id=decoded.user_id,
event_type=SecurityEventType.DB_VIEWER_ACCESSED,
ip_address=get_client_ip(request, settings),
)
await db.commit()
session_token = create_db_viewer_session_token(decoded.user_id, settings)
# Adminer's own driver dropdown defaults to MySQL, not Postgres - a bare
# "/" redirect leaves it selecting MySQL against a host that only speaks
# Postgres, producing a "Connection refused" error that has nothing to
# do with this app's own auth. `?pgsql=postgres` pre-selects the right
# driver and server (the internal Compose service name, same in dev and
# prod) - only the Postgres password itself is left for the admin to type.
response = RedirectResponse(url="/?pgsql=postgres", status_code=status.HTTP_302_FOUND)
response.set_cookie(
SESSION_COOKIE_NAME,
session_token,
max_age=60 * 60,
httponly=True,
secure=True,
samesite="lax",
# No `domain=` - defaults to the exact host (db.<domain>), not
# shared with ciagent.org/api.ciagent.org/git.ciagent.org.
)
return response
@router.get("/verify")
async def verify_db_viewer_session(
request: Request,
settings: Settings = Depends(get_settings),
db: AsyncSession = Depends(get_db),
) -> Response:
"""Reached only via Nginx's `auth_request` subrequest (internal `/_verify`
location) - carries just the forwarded Cookie header, no Authorization.
Re-checks `is_admin` fresh from the database on every call, not just at
token-mint time, so revoking an admin takes effect on their very next
request here - same live-check behavior as `require_admin` elsewhere."""
token = request.cookies.get(SESSION_COOKIE_NAME)
if token is None:
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="No session")
try:
decoded = decode_token(token, settings, TokenType.DB_VIEWER_SESSION)
except InvalidTokenError as exc:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid or expired session"
) from exc
user = await UserRepository(db).get_by_id(decoded.user_id)
if user is None or not user.is_admin:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED, detail="Admin privileges required"
)
return Response(status_code=status.HTTP_200_OK)
+2
View File
@@ -11,6 +11,7 @@ from app.api.v1 import (
auth, auth,
companies, companies,
dashboard, dashboard,
db_viewer,
monitoring, monitoring,
notification_destinations, notification_destinations,
reports, reports,
@@ -24,6 +25,7 @@ api_v1_router = APIRouter(prefix="/api/v1")
api_v1_router.include_router(system.router) api_v1_router.include_router(system.router)
api_v1_router.include_router(auth.router) api_v1_router.include_router(auth.router)
api_v1_router.include_router(admin.router) api_v1_router.include_router(admin.router)
api_v1_router.include_router(db_viewer.router)
api_v1_router.include_router(user_api_keys.router) api_v1_router.include_router(user_api_keys.router)
api_v1_router.include_router(companies.router) api_v1_router.include_router(companies.router)
api_v1_router.include_router(notification_destinations.router) api_v1_router.include_router(notification_destinations.router)
+40 -2
View File
@@ -7,11 +7,12 @@ ready to serve traffic".
from __future__ import annotations from __future__ import annotations
import uuid
from typing import Literal from typing import Literal
import httpx import httpx
import redis.asyncio as redis_asyncio import redis.asyncio as redis_asyncio
from fastapi import APIRouter, Depends, Request from fastapi import APIRouter, Depends, Request, status
from pydantic import BaseModel from pydantic import BaseModel
from sqlalchemy import text from sqlalchemy import text
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
@@ -23,8 +24,9 @@ from app.core.security import get_client_ip, is_localhost
from app.db.session import get_db from app.db.session import get_db
from app.models.enums import SystemSecretKey from app.models.enums import SystemSecretKey
from app.models.user import User from app.models.user import User
from app.schemas.provisioning import PendingProvisioningCreate, PendingProvisioningResponse
from app.schemas.system_secret import SetSystemSecretRequest, SystemSecretStatus from app.schemas.system_secret import SetSystemSecretRequest, SystemSecretStatus
from app.services import system_secret_service from app.services import provisioning_service, system_secret_service
from app.services.enrichment_service import estimate_max_credits_per_company from app.services.enrichment_service import estimate_max_credits_per_company
logger = get_logger(__name__) logger = get_logger(__name__)
@@ -194,3 +196,39 @@ async def system_logs(
"""Most-recent-first view into the application's live log stream (capped """Most-recent-first view into the application's live log stream (capped
at the last 500 entries app-wide, see `core/logging.py`).""" at the last 500 entries app-wide, see `core/logging.py`)."""
return [LogEntryResponse(**entry) for entry in await get_recent_logs(settings, limit=100)] return [LogEntryResponse(**entry) for entry in await get_recent_logs(settings, limit=100)]
@router.get("/system/pending-provisioning", response_model=list[PendingProvisioningResponse])
async def list_pending_provisioning(
db: AsyncSession = Depends(get_db), _admin: User = Depends(require_admin)
) -> list[PendingProvisioningResponse]:
"""Queued not-yet-existing accounts - see app.services.provisioning_service."""
return [
PendingProvisioningResponse.model_validate(p, from_attributes=True)
for p in await provisioning_service.list_pending(db)
]
@router.post(
"/system/pending-provisioning",
response_model=PendingProvisioningResponse,
status_code=status.HTTP_201_CREATED,
)
async def create_pending_provisioning(
payload: PendingProvisioningCreate,
db: AsyncSession = Depends(get_db),
_admin: User = Depends(require_admin),
) -> PendingProvisioningResponse:
record = await provisioning_service.create_pending(
db, payload.email, payload.source_user_id, payload.make_admin
)
return PendingProvisioningResponse.model_validate(record, from_attributes=True)
@router.delete("/system/pending-provisioning/{pending_id}", status_code=status.HTTP_204_NO_CONTENT)
async def delete_pending_provisioning(
pending_id: uuid.UUID,
db: AsyncSession = Depends(get_db),
_admin: User = Depends(require_admin),
) -> None:
await provisioning_service.delete_pending(db, pending_id)
+14
View File
@@ -29,6 +29,15 @@ from app.models.user import User
from app.repositories.user_repository import UserRepository from app.repositories.user_repository import UserRepository
from app.services.auth_service import get_or_create_local_user from app.services.auth_service import get_or_create_local_user
# Everything an account with must_change_password=True can still reach -
# just enough to discover the flag (/me), fix it (/change-password), and
# bail out (/logout, which doesn't even route through get_current_user but
# is listed for clarity). Every other endpoint 403s until they change it -
# see app.services.auth_service.require_password_change.
_PASSWORD_CHANGE_EXEMPT_PATHS = frozenset(
{"/api/v1/auth/me", "/api/v1/auth/change-password", "/api/v1/auth/logout"}
)
async def get_current_user( async def get_current_user(
request: Request, request: Request,
@@ -58,6 +67,11 @@ async def get_current_user(
status_code=status.HTTP_401_UNAUTHORIZED, status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid or expired access token", detail="Invalid or expired access token",
) )
if user.must_change_password and request.url.path not in _PASSWORD_CHANGE_EXEMPT_PATHS:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Password change required",
)
return user return user
+13
View File
@@ -75,6 +75,11 @@ def verify_password(raw_password: str, password_hash: str) -> bool:
class TokenType(StrEnum): class TokenType(StrEnum):
ACCESS = "access" ACCESS = "access"
REFRESH = "refresh" REFRESH = "refresh"
# Settings -> Database viewer: a short-lived token minted from a live
# admin session, exchanged (via Nginx's db.ciagent.org /_auth route) for
# a longer-lived DB_VIEWER_SESSION cookie. See app/api/v1/db_viewer.py.
DB_VIEWER_BOOTSTRAP = "db_viewer_bootstrap"
DB_VIEWER_SESSION = "db_viewer_session"
@dataclass(frozen=True) @dataclass(frozen=True)
@@ -105,6 +110,14 @@ def create_refresh_token(user_id: uuid.UUID, settings: Settings) -> tuple[str, s
return token, jti, expires_at return token, jti, expires_at
def create_db_viewer_bootstrap_token(user_id: uuid.UUID, settings: Settings) -> str:
return _encode_token(user_id, TokenType.DB_VIEWER_BOOTSTRAP, timedelta(minutes=2), settings)
def create_db_viewer_session_token(user_id: uuid.UUID, settings: Settings) -> str:
return _encode_token(user_id, TokenType.DB_VIEWER_SESSION, timedelta(minutes=60), settings)
def _encode_token( def _encode_token(
user_id: uuid.UUID, user_id: uuid.UUID,
token_type: TokenType, token_type: TokenType,
+1
View File
@@ -22,6 +22,7 @@ from app.models.notification_destination import ( # noqa: F401
NotificationDestinationCompany, NotificationDestinationCompany,
) )
from app.models.password_history import PasswordHistoryEntry # noqa: F401 from app.models.password_history import PasswordHistoryEntry # noqa: F401
from app.models.pending_provisioning import PendingProvisioning # noqa: F401
from app.models.refresh_token import RefreshToken # noqa: F401 from app.models.refresh_token import RefreshToken # noqa: F401
from app.models.report import Report # noqa: F401 from app.models.report import Report # noqa: F401
from app.models.snapshot import Snapshot # noqa: F401 from app.models.snapshot import Snapshot # noqa: F401
+4
View File
@@ -153,6 +153,10 @@ class SecurityEventType(StrEnum):
EMAIL_VERIFIED = "email_verified" EMAIL_VERIFIED = "email_verified"
SERVER_SECRET_UPDATED = "server_secret_updated" SERVER_SECRET_UPDATED = "server_secret_updated"
API_KEY_UPDATED = "api_key_updated" API_KEY_UPDATED = "api_key_updated"
DB_VIEWER_SESSION_CREATED = "db_viewer_session_created"
DB_VIEWER_ACCESSED = "db_viewer_accessed"
PASSWORD_CHANGED = "password_changed"
PASSWORD_CHANGE_REQUIRED = "password_change_required"
class ApiKeyProvider(StrEnum): class ApiKeyProvider(StrEnum):
@@ -0,0 +1,31 @@
"""Lets an admin pre-configure what a not-yet-existing account should get
the moment someone verifies their email with a matching address - copies
of another user's per-user API keys and companies (with their full data:
sources, enrichment, reports, snapshots, etc.), an email notification
destination for the new account's own address, and optionally an admin
promotion. Built for demoing the app to people who don't have accounts
yet without hand-entering everything for them after the fact.
Applied at email-verification time, not raw registration, since
registering an email address doesn't prove you own it - see
auth_service.verify_email / provisioning_service.apply_if_pending.
"""
from __future__ import annotations
import uuid
from sqlalchemy import Boolean, ForeignKey, String
from sqlalchemy.orm import Mapped, mapped_column
from app.db.base import Base, TimestampMixin, UUIDPrimaryKeyMixin
class PendingProvisioning(Base, UUIDPrimaryKeyMixin, TimestampMixin):
__tablename__ = "pending_provisionings"
email: Mapped[str] = mapped_column(String(320), unique=True, index=True)
source_user_id: Mapped[uuid.UUID] = mapped_column(
ForeignKey("users.id", ondelete="CASCADE"), index=True
)
make_admin: Mapped[bool] = mapped_column(Boolean, default=False)
+6
View File
@@ -42,6 +42,12 @@ class User(Base, UUIDPrimaryKeyMixin, TimestampMixin):
failed_login_count: Mapped[int] = mapped_column(Integer, default=0) failed_login_count: Mapped[int] = mapped_column(Integer, default=0)
locked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True) locked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
# Admin-forced reset (e.g. before handing a demo account to someone) -
# see app.auth.dependencies.get_current_user, which blocks every
# endpoint except /auth/me, /auth/change-password, and /auth/logout
# while this is true.
must_change_password: Mapped[bool] = mapped_column(Boolean, default=False)
refresh_tokens: Mapped[list[RefreshToken]] = relationship( refresh_tokens: Mapped[list[RefreshToken]] = relationship(
back_populates="user", cascade="all, delete-orphan" back_populates="user", cascade="all, delete-orphan"
) )
+59 -10
View File
@@ -21,17 +21,66 @@ SYSTEM_PROMPT = (
"third-party data provider at onboarding, also real evidence, not your own knowledge), " "third-party data provider at onboarding, also real evidence, not your own knowledge), "
"stored source documents, and previously detected changes. Never introduce " "stored source documents, and previously detected changes. Never introduce "
"facts from outside knowledge, even if you recognize the company - if it isn't in the " "facts from outside knowledge, even if you recognize the company - if it isn't in the "
"evidence block, it doesn't go in the report. The company_profile and company_enrichment " "evidence block, it doesn't go in the report.\n\n"
"fields ARE real evidence and should ground company_overview/market_positioning/" "company_profile and company_enrichment are real evidence and must ground every "
"financial_signals/leadership_changes/products_and_services/competitor_comparison/" "applicable section even when source_documents and detected_changes are sparse or "
"customer_sentiment/etc even when source_documents and detected_changes are sparse or "
"empty - do not say 'insufficient evidence' for a field company_profile or " "empty - do not say 'insufficient evidence' for a field company_profile or "
"company_enrichment already answers. Every finding must be " "company_enrichment already answers. In particular, the list-of-Finding sections are "
"traceable to the evidence given and must carry an honest confidence label: confirmed " "NOT limited to newly detected changes - they must also represent the company's "
"(the source states it directly), strongly_indicated, likely, possible, unconfirmed, or " "current, confirmed state whenever company_enrichment/company_profile answers them "
"insufficient_evidence. When evidence is thin or missing for a section, say so explicitly " "directly, the same way you would use that evidence for a prose field:\n"
"in that section rather than inventing content. Distinguish clearly between what a source " "- products_and_services: if company_enrichment.products is present, emit ONE Finding "
"states and what you are inferring." "per product (headline = product name, summary = its description/category, "
"confidence = confirmed). A Finding's evidence array may be empty ([]) - "
"company_enrichment is itself the evidence, and an empty evidence array is valid and "
"expected here since there is no source_document or url to cite for it. For example, "
"given company_enrichment.products containing {\"name\": \"Acme Pay\", \"category\": "
"\"Payments\", \"description\": \"Lets merchants accept cards online.\"}, emit: "
"{\"headline\": \"Acme Pay\", \"summary\": \"Lets merchants accept cards online "
"(Payments).\", \"confidence\": \"confirmed\", \"evidence\": []}. Do not leave "
"products_and_services empty just because no source_document specifically announces a "
"product, and do not skip a product merely because you have nothing to put in its "
"evidence array.\n"
"- recent_developments: if company_enrichment.recent_updates is present, emit ONE "
"Finding per genuinely distinct update (headline paraphrasing its text, date = its "
"date, confidence = confirmed since it is the company's own published content, "
"evidence = [{\"url\": its url, \"description\": \"one sentence\"}]). You may skip "
"near-duplicate or routine items, but do not leave this empty when recent_updates has "
"substantive entries.\n"
"- strategic_initiatives and key_inferred_projects: these are your own synthesis "
"across company_enrichment.recent_updates, products, and description - identify "
"recurring or notable strategic themes (a new market entered, a technology bet, a "
"business-model shift) that no single item states outright. Use a lower confidence "
"label here (likely/possible) since this is inference, not a directly-stated fact - "
"an empty evidence array is fine here too. Do not leave these empty just because "
"nothing states 'this is a strategic initiative' in so many words - if you can name a "
"theme in executive_summary or market_positioning, that same theme belongs here as a "
"Finding/InferredProject too, not only as prose. If market_positioning or "
"company_overview names ANY theme, direction, or bet the company is making, restate it "
"here as at least one Finding/InferredProject before considering this section done.\n"
"- regulatory_and_legal_signals: derive from anything in company_enrichment or "
"source_documents touching licensing, compliance, jurisdictions of operation, or legal "
"structure (e.g. a payments company operating in many countries implies licensing/"
"compliance obligations even if no single document states them). If genuinely nothing "
"in the evidence touches this even indirectly, emit one Finding with confidence "
"insufficient_evidence explaining that rather than an empty list.\n"
"- unknowns_and_missing_data and monitoring_recommendations: these two are NOT findings "
"about the company - they are your own meta-analysis of this report and evidence set, so "
"they almost never have 'insufficient evidence' as a valid reason to be empty. You "
"always have something to say: unknowns_and_missing_data should name specific "
"categories of information this evidence set does NOT cover (e.g. 'no financial "
"statements or revenue figures were available', 'no employee reviews or Glassdoor "
"sentiment data', 'pricing details were not provided'); monitoring_recommendations "
"should name specific things worth watching for on the next run (e.g. 'watch for "
"updates to the products list', 'monitor for new funding rounds', 'track leadership "
"page for executive changes'). Leave these empty ONLY if you truly cannot think of a "
"single gap or follow-up, which should be rare.\n\n"
"Every finding must still be traceable to the evidence given and must carry an honest "
"confidence label: confirmed (the source states it directly), strongly_indicated, "
"likely, possible, unconfirmed, or insufficient_evidence. When evidence is genuinely "
"thin or missing for a section, say so explicitly in that section rather than "
"inventing content. Distinguish clearly between what a source states and what you are "
"inferring."
) )
@@ -0,0 +1,37 @@
from __future__ import annotations
import uuid
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.pending_provisioning import PendingProvisioning
class PendingProvisioningRepository:
def __init__(self, db: AsyncSession) -> None:
self.db = db
async def list_all(self) -> list[PendingProvisioning]:
result = await self.db.execute(select(PendingProvisioning))
return list(result.scalars().all())
async def get_by_email(self, email: str) -> PendingProvisioning | None:
result = await self.db.execute(
select(PendingProvisioning).where(PendingProvisioning.email == email.lower())
)
return result.scalar_one_or_none()
async def create(
self, email: str, source_user_id: uuid.UUID, make_admin: bool
) -> PendingProvisioning:
record = PendingProvisioning(
email=email.lower(), source_user_id=source_user_id, make_admin=make_admin
)
self.db.add(record)
await self.db.flush()
return record
async def delete(self, record: PendingProvisioning) -> None:
await self.db.delete(record)
await self.db.flush()
+14 -1
View File
@@ -2,7 +2,7 @@ from __future__ import annotations
import uuid import uuid
from sqlalchemy import select from sqlalchemy import func, select
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from app.models.user import User from app.models.user import User
@@ -15,6 +15,12 @@ class UserRepository:
async def get_by_id(self, user_id: uuid.UUID) -> User | None: async def get_by_id(self, user_id: uuid.UUID) -> User | None:
return await self.db.get(User, user_id) return await self.db.get(User, user_id)
async def count_admins(self) -> int:
result = await self.db.execute(
select(func.count()).select_from(User).where(User.is_admin.is_(True))
)
return result.scalar_one()
async def get_by_email(self, email: str) -> User | None: async def get_by_email(self, email: str) -> User | None:
result = await self.db.execute(select(User).where(User.email == email.lower())) result = await self.db.execute(select(User).where(User.email == email.lower()))
return result.scalar_one_or_none() return result.scalar_one_or_none()
@@ -23,6 +29,13 @@ class UserRepository:
result = await self.db.execute(select(User.email).where(User.is_admin.is_(True))) result = await self.db.execute(select(User.email).where(User.is_admin.is_(True)))
return list(result.scalars().all()) return list(result.scalars().all())
async def delete(self, user: User) -> None:
"""Cascades (ON DELETE CASCADE, see migrations) to every row the
user owns - companies and everything under them, refresh tokens,
security events, etc. Irreversible."""
await self.db.delete(user)
await self.db.flush()
async def create( async def create(
self, self,
*, *,
+18
View File
@@ -73,6 +73,24 @@ class ConfirmPasswordResetRequest(BaseModel):
return _validate_password_strength(value) return _validate_password_strength(value)
class DeleteAccountRequest(BaseModel):
password: str = Field(min_length=1, max_length=128)
class ChangePasswordRequest(BaseModel):
current_password: str = Field(min_length=1, max_length=128)
new_password: str = Field(min_length=10, max_length=128)
@field_validator("new_password")
@classmethod
def _password_strength(cls, value: str) -> str:
return _validate_password_strength(value)
class RequirePasswordChangeRequest(BaseModel):
email: EmailStr
class SecurityEventResponse(BaseModel): class SecurityEventResponse(BaseModel):
event_type: str event_type: str
ip_address: str ip_address: str
+20
View File
@@ -0,0 +1,20 @@
from __future__ import annotations
import uuid
from datetime import datetime
from pydantic import BaseModel, EmailStr
class PendingProvisioningCreate(BaseModel):
email: EmailStr
source_user_id: uuid.UUID
make_admin: bool = False
class PendingProvisioningResponse(BaseModel):
id: uuid.UUID
email: str
source_user_id: uuid.UUID
make_admin: bool
created_at: datetime
+1
View File
@@ -14,6 +14,7 @@ class UserResponse(BaseModel):
timezone: str timezone: str
is_active: bool is_active: bool
is_admin: bool is_admin: bool
must_change_password: bool
class MeResponse(UserResponse): class MeResponse(UserResponse):
+100 -2
View File
@@ -13,7 +13,13 @@ from datetime import UTC, datetime, timedelta
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from app.core.config import Settings from app.core.config import Settings
from app.core.errors import AuthenticationError, ConflictError, ThrottledError, ValidationAppError from app.core.errors import (
AuthenticationError,
ConflictError,
NotFoundError,
ThrottledError,
ValidationAppError,
)
from app.core.security import ( from app.core.security import (
InvalidTokenError, InvalidTokenError,
TokenType, TokenType,
@@ -37,6 +43,7 @@ from app.repositories.user_known_ip_repository import UserKnownIpRepository
from app.repositories.user_repository import UserRepository from app.repositories.user_repository import UserRepository
from app.repositories.user_security_event_repository import UserSecurityEventRepository from app.repositories.user_security_event_repository import UserSecurityEventRepository
from app.schemas.auth import ( from app.schemas.auth import (
ChangePasswordRequest,
ConfirmPasswordResetRequest, ConfirmPasswordResetRequest,
LoginRequest, LoginRequest,
RegisterRequest, RegisterRequest,
@@ -45,7 +52,7 @@ from app.schemas.auth import (
TokenResponse, TokenResponse,
VerifyEmailRequest, VerifyEmailRequest,
) )
from app.services import ip_throttle_service, security_email_service from app.services import ip_throttle_service, provisioning_service, security_email_service
EMAIL_CODE_VALID_HOURS = 36 EMAIL_CODE_VALID_HOURS = 36
@@ -118,11 +125,25 @@ async def register(
if existing is not None: if existing is not None:
raise ConflictError("An account with this email already exists") raise ConflictError("An account with this email already exists")
# Bootstraps admin access on a fresh deployment - otherwise the only way
# to ever get an admin account is direct DB access, which is a real
# chicken-and-egg problem for anyone self-hosting from a clean clone.
# Checked by admin *count*, not total user count, so this also
# self-heals if the last admin ever deletes their own account (Settings
# -> Delete account) - the next registration becomes admin again rather
# than leaving the deployment permanently admin-less. A benign race is
# possible if two people register in the same instant on a brand-new,
# zero-admin deployment (both could become admin) - acceptable for a
# bootstrapping check that only ever matters once, before any real
# traffic exists.
is_first_admin = await repo.count_admins() == 0
user = await repo.create( user = await repo.create(
email=payload.email, email=payload.email,
password_hash=hash_password(payload.password), password_hash=hash_password(payload.password),
display_name=payload.display_name, display_name=payload.display_name,
timezone=payload.timezone, timezone=payload.timezone,
is_admin=is_first_admin,
# Test suite has no inbox to read a real code from - same # Test suite has no inbox to read a real code from - same
# app_env == "test" precedent already used to disable rate limiting # app_env == "test" precedent already used to disable rate limiting
# (app/core/rate_limit.py). The code-generation/sending/throttle # (app/core/rate_limit.py). The code-generation/sending/throttle
@@ -181,6 +202,11 @@ async def verify_email(db: AsyncSession, client_ip: str, payload: VerifyEmailReq
await code_repo.mark_used(record) await code_repo.mark_used(record)
user.email_verified = True user.email_verified = True
# Verifying an email is the point at which the app can actually trust
# someone owns this address - if an admin queued this address up for
# pre-provisioning (see provisioning_service), this is where it fires,
# never at raw registration (which proves nothing about ownership).
await provisioning_service.apply_if_pending(db, user)
await ip_throttle_service.reset_on_success(db, client_ip, ThrottleAction.VERIFY_EMAIL_CODE) await ip_throttle_service.reset_on_success(db, client_ip, ThrottleAction.VERIFY_EMAIL_CODE)
await UserSecurityEventRepository(db).create( await UserSecurityEventRepository(db).create(
user_id=user.id, event_type=SecurityEventType.EMAIL_VERIFIED, ip_address=client_ip user_id=user.id, event_type=SecurityEventType.EMAIL_VERIFIED, ip_address=client_ip
@@ -426,3 +452,75 @@ async def list_security_events(db: AsyncSession, user_id: uuid.UUID) -> list[Use
"""The calling user's own security activity - the user-facing """The calling user's own security activity - the user-facing
counterpart to the admin-only app-wide log feed (core/logging.py).""" counterpart to the admin-only app-wide log feed (core/logging.py)."""
return await UserSecurityEventRepository(db).list_for_user(user_id) return await UserSecurityEventRepository(db).list_for_user(user_id)
async def delete_account(db: AsyncSession, user: User, password: str) -> None:
"""Permanently deletes the account and, via ON DELETE CASCADE foreign
keys (see migrations), everything it owns - companies and everything
under them, refresh tokens, security events, API keys, etc. There is no
soft-delete or recovery path. Rejected for accounts with no
password_hash (the fixed AUTH_MODE=local user) - there's nothing to
verify the caller's identity against."""
if user.password_hash is None:
raise ValidationAppError("Account deletion isn't available for this account.")
if not verify_password(password, user.password_hash):
raise AuthenticationError("Incorrect password")
await UserRepository(db).delete(user)
await db.commit()
async def change_password(
db: AsyncSession,
settings: Settings,
client_ip: str,
user: User,
payload: ChangePasswordRequest,
) -> TokenResponse:
"""Authenticated in-app password change - distinct from
confirm_password_reset (which is the emailed-code flow for someone who
can't log in at all). Also the only way to clear must_change_password,
the admin-forced-reset flag (see require_password_change below)."""
if user.password_hash is None:
raise ValidationAppError("This account doesn't use password sign-in.")
if not verify_password(payload.current_password, user.password_hash):
raise AuthenticationError("Incorrect current password")
history_repo = PasswordHistoryRepository(db)
previous_hashes = await history_repo.list_hashes_for_user(user.id)
previous_hashes.append(user.password_hash)
if any(verify_password(payload.new_password, h) for h in previous_hashes):
raise ValidationAppError("You've used this password before. Choose a different one.")
await history_repo.add(user_id=user.id, password_hash=user.password_hash)
user.password_hash = hash_password(payload.new_password)
user.must_change_password = False
# Rotate every session, including the one making this request - the
# fresh token pair returned below replaces it immediately, so the
# caller keeps working without a forced re-login.
await RefreshTokenRepository(db).revoke_all_for_user(user.id)
await UserSecurityEventRepository(db).create(
user_id=user.id, event_type=SecurityEventType.PASSWORD_CHANGED, ip_address=client_ip
)
return await _issue_token_pair(db, settings, user)
async def require_password_change(db: AsyncSession, client_ip: str, email: str) -> User:
"""Admin action: flags an account so its next request is blocked
everywhere except /auth/me, /auth/change-password, and /auth/logout
(enforced in app.auth.dependencies.get_current_user) until they set a
new password. Meant for handing a demo account to someone with a known
sample password."""
user = await UserRepository(db).get_by_email(email)
if user is None:
raise NotFoundError("No account with that email")
if user.password_hash is None:
raise ValidationAppError("This account doesn't use password sign-in.")
user.must_change_password = True
await UserSecurityEventRepository(db).create(
user_id=user.id,
event_type=SecurityEventType.PASSWORD_CHANGE_REQUIRED,
ip_address=client_ip,
)
await db.commit()
return user
+102
View File
@@ -20,6 +20,7 @@ from datetime import UTC, datetime
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from app.core.config import Settings from app.core.config import Settings
from app.core.errors import NotFoundError
from app.core.logging import get_logger from app.core.logging import get_logger
from app.enrichment.base import EnrichmentProvider from app.enrichment.base import EnrichmentProvider
from app.models.company import Company from app.models.company import Company
@@ -174,3 +175,104 @@ async def enrich_company(
failed_sections=list(errors.keys()), failed_sections=list(errors.keys()),
) )
return enrichment return enrichment
# Every entry here maps 1:1 onto one of NinjaPear's independent
# per-company endpoints (see app/enrichment/ninjapear.py) - refreshing one
# section never re-fetches or touches any of the others.
REFRESHABLE_SECTIONS = ("details", "funding", "updates", "competitors", "products", "customers")
async def refresh_section(
db: AsyncSession,
settings: Settings,
provider: EnrichmentProvider,
company: Company,
section: str,
) -> CompanyEnrichment:
"""Re-runs a single NinjaPear endpoint for a company that already has
an enrichment record but came back empty or failed for just this one
section (e.g. `products` returning `[]` while everything else
succeeded - a real, honest gap in NinjaPear's own coverage for that
company, not a bug in this app). Only ever replaces this section's own
slice of `data`/`errors` - every other section's stored data is left
exactly as it was."""
if section not in REFRESHABLE_SECTIONS:
raise ValueError(f"Unknown enrichment section: {section}")
repo = CompanyEnrichmentRepository(db)
existing = await repo.get_for_company(company.id)
if existing is None:
raise NotFoundError("Company has no enrichment record to refresh")
website = company.official_website
data = dict(existing.data)
errors = dict(existing.errors)
credits_spent = existing.credits_spent or 0
try:
if section == "details":
details = await provider.get_company_details(company.name, website)
data["employee_count"] = details.employee_count_range
data["description"] = details.description
data["industry"] = details.industry
data["founded_year"] = details.founded_year
data["specialties"] = details.specialties
new_leadership = [m.model_dump() for m in details.leadership_team]
# A details refresh re-fetches the leadership roster itself,
# but not the separate per-leader work_email/person_profile
# lookups - preserve those by matching on name so a refresh
# never regresses contact info that was already found.
old_by_name = {
m.get("name"): m for m in (data.get("leadership_team") or []) if m.get("name")
}
for member in new_leadership:
old = old_by_name.get(member.get("name"))
if old:
member["work_email"] = member.get("work_email") or old.get("work_email")
member["profile_url"] = member.get("profile_url") or old.get("profile_url")
member["bio"] = member.get("bio") or old.get("bio")
data["leadership_team"] = new_leadership
elif section == "funding":
funding = await provider.get_funding(company.name, website)
data["funding"] = funding.model_dump()
elif section == "updates":
updates = await provider.get_updates(company.name, website)
data["recent_updates"] = [u.model_dump() for u in updates]
elif section == "competitors":
competitors = await provider.get_competitors(company.name, website)
data["competitors"] = [c.model_dump() for c in competitors]
elif section == "products":
products = await provider.get_products(company.name, website)
data["products"] = [p.model_dump() for p in products]
elif section == "customers":
customers = await provider.get_customers(company.name, website)
data["customers"] = [c.model_dump() for c in customers]
errors.pop(section, None)
credits_spent += _CREDIT_COSTS.get(section, 0)
except Exception as exc: # noqa: BLE001 - surfaced via `errors`, same as the initial run
errors[section] = str(exc)
logger.warning(
"enrichment_section_refresh_failed",
section=section,
company_id=str(company.id),
error=str(exc),
)
new_status = EnrichmentStatus.PARTIAL if errors else EnrichmentStatus.COMPLETE
enrichment = await repo.upsert(
company.id,
status=new_status,
data=data,
errors=errors,
credits_spent=credits_spent,
fetched_at=datetime.now(UTC),
)
await db.commit()
logger.info(
"company_enrichment_section_refreshed",
company_id=str(company.id),
section=section,
succeeded=section not in errors,
)
return enrichment
@@ -0,0 +1,295 @@
"""Pre-provisions a not-yet-existing account: the moment someone verifies
an email address an admin has queued up (see PendingProvisioning), they
get a deep copy of another user's per-user API keys and companies -
including every tab's worth of data (sources, source documents, monitoring
runs, reports, snapshots, detected changes, enrichment) - plus an email
notification destination for their own address linked to the copied
companies, and optionally an admin promotion.
Applied at email-verification time (see auth_service.verify_email), not
raw registration - registering an email doesn't prove ownership of it,
verifying does.
"""
from __future__ import annotations
import uuid
from typing import Any
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.errors import ConflictError, NotFoundError
from app.models.company import Company, CompanyAlias, Competitor
from app.models.company_enrichment import CompanyEnrichment
from app.models.detected_change import DetectedChange
from app.models.enums import NotificationType, SeverityLevel
from app.models.monitor_configuration import MonitorConfiguration
from app.models.monitoring_run import MonitoringRun
from app.models.notification_destination import (
NotificationDestination,
NotificationDestinationCompany,
)
from app.models.pending_provisioning import PendingProvisioning
from app.models.report import Report
from app.models.snapshot import Snapshot
from app.models.source import Source
from app.models.source_document import SourceDocument
from app.models.user import User
from app.models.user_api_key import UserApiKey
from app.repositories.pending_provisioning_repository import PendingProvisioningRepository
from app.repositories.user_repository import UserRepository
def _copy_row[ModelT](
model_cls: type[ModelT], source_row: Any, overrides: dict[str, Any]
) -> ModelT:
"""A new ORM instance of the same class, with every column value copied
from `source_row` except whatever `overrides` replaces (always at least
`id`, plus any foreign keys that need remapping to the new owner's
copies of their parents). Generic over every table involved so adding a
new company-related table later doesn't require touching this file."""
data = {
col.name: getattr(source_row, col.name) for col in model_cls.__table__.columns # type: ignore[attr-defined]
}
data.update(overrides)
return model_cls(**data)
async def _clone_company(
db: AsyncSession, source_company_id: uuid.UUID, target_user_id: uuid.UUID
) -> Company:
id_map: dict[uuid.UUID, uuid.UUID] = {}
def new_id(old_id: uuid.UUID) -> uuid.UUID:
if old_id not in id_map:
id_map[old_id] = uuid.uuid4()
return id_map[old_id]
company = await db.get(Company, source_company_id)
assert company is not None
new_company = _copy_row(Company, company, {"id": new_id(company.id), "user_id": target_user_id})
db.add(new_company)
await db.flush()
for alias in (
await db.execute(select(CompanyAlias).where(CompanyAlias.company_id == source_company_id))
).scalars():
db.add(_copy_row(CompanyAlias, alias, {"id": uuid.uuid4(), "company_id": new_company.id}))
for competitor in (
await db.execute(select(Competitor).where(Competitor.company_id == source_company_id))
).scalars():
db.add(
_copy_row(Competitor, competitor, {"id": uuid.uuid4(), "company_id": new_company.id})
)
config = (
await db.execute(
select(MonitorConfiguration).where(MonitorConfiguration.company_id == source_company_id)
)
).scalar_one_or_none()
if config is not None:
db.add(
_copy_row(
MonitorConfiguration, config, {"id": uuid.uuid4(), "company_id": new_company.id}
)
)
enrichment = (
await db.execute(
select(CompanyEnrichment).where(CompanyEnrichment.company_id == source_company_id)
)
).scalar_one_or_none()
if enrichment is not None:
db.add(
_copy_row(
CompanyEnrichment, enrichment, {"id": uuid.uuid4(), "company_id": new_company.id}
)
)
sources = (
(await db.execute(select(Source).where(Source.company_id == source_company_id)))
.scalars()
.all()
)
for source in sources:
db.add(_copy_row(Source, source, {"id": new_id(source.id), "company_id": new_company.id}))
await db.flush()
for doc in (
await db.execute(
select(SourceDocument).where(SourceDocument.company_id == source_company_id)
)
).scalars():
db.add(
_copy_row(
SourceDocument,
doc,
{
"id": uuid.uuid4(),
"company_id": new_company.id,
"source_id": new_id(doc.source_id),
},
)
)
runs = (
(
await db.execute(
select(MonitoringRun).where(MonitoringRun.company_id == source_company_id)
)
)
.scalars()
.all()
)
for run in runs:
db.add(_copy_row(MonitoringRun, run, {"id": new_id(run.id), "company_id": new_company.id}))
await db.flush()
for report in (
await db.execute(select(Report).where(Report.company_id == source_company_id))
).scalars():
db.add(
_copy_row(
Report,
report,
{
"id": uuid.uuid4(),
"company_id": new_company.id,
"monitoring_run_id": (
new_id(report.monitoring_run_id) if report.monitoring_run_id else None
),
},
)
)
snapshots = (
(await db.execute(select(Snapshot).where(Snapshot.company_id == source_company_id)))
.scalars()
.all()
)
for snapshot in snapshots:
db.add(
_copy_row(
Snapshot,
snapshot,
{
"id": new_id(snapshot.id),
"company_id": new_company.id,
"source_id": new_id(snapshot.source_id) if snapshot.source_id else None,
"monitoring_run_id": (
new_id(snapshot.monitoring_run_id) if snapshot.monitoring_run_id else None
),
},
)
)
await db.flush()
for change in (
await db.execute(
select(DetectedChange).where(DetectedChange.company_id == source_company_id)
)
).scalars():
db.add(
_copy_row(
DetectedChange,
change,
{
"id": uuid.uuid4(),
"company_id": new_company.id,
"source_id": new_id(change.source_id) if change.source_id else None,
"monitoring_run_id": (
new_id(change.monitoring_run_id) if change.monitoring_run_id else None
),
"previous_snapshot_id": (
new_id(change.previous_snapshot_id) if change.previous_snapshot_id else None
),
"current_snapshot_id": (
new_id(change.current_snapshot_id) if change.current_snapshot_id else None
),
},
)
)
return new_company
async def _copy_api_keys(
db: AsyncSession, source_user_id: uuid.UUID, target_user_id: uuid.UUID
) -> None:
keys = (
(await db.execute(select(UserApiKey).where(UserApiKey.user_id == source_user_id)))
.scalars()
.all()
)
for key in keys:
db.add(_copy_row(UserApiKey, key, {"id": uuid.uuid4(), "user_id": target_user_id}))
async def create_pending(
db: AsyncSession, email: str, source_user_id: uuid.UUID, make_admin: bool
) -> PendingProvisioning:
if await UserRepository(db).get_by_email(email) is not None:
raise ConflictError("An account with this email already exists")
if await PendingProvisioningRepository(db).get_by_email(email) is not None:
raise ConflictError("This email is already queued for provisioning")
record = await PendingProvisioningRepository(db).create(email, source_user_id, make_admin)
await db.commit()
return record
async def list_pending(db: AsyncSession) -> list[PendingProvisioning]:
return await PendingProvisioningRepository(db).list_all()
async def delete_pending(db: AsyncSession, pending_id: uuid.UUID) -> None:
repo = PendingProvisioningRepository(db)
match = next((p for p in await repo.list_all() if p.id == pending_id), None)
if match is None:
raise NotFoundError("Pending provisioning entry not found")
await repo.delete(match)
await db.commit()
async def apply_if_pending(db: AsyncSession, new_user: User) -> bool:
"""Called right after a fresh account's email gets verified. Returns
True if a pending provisioning entry matched and was applied (and
consumed - a pending entry only ever fires once)."""
pending = await PendingProvisioningRepository(db).get_by_email(new_user.email)
if pending is None:
return False
await _copy_api_keys(db, pending.source_user_id, new_user.id)
source_companies = (
(await db.execute(select(Company).where(Company.user_id == pending.source_user_id)))
.scalars()
.all()
)
new_company_ids = [
(await _clone_company(db, company.id, new_user.id)).id for company in source_companies
]
if new_company_ids:
destination = NotificationDestination(
user_id=new_user.id,
type=NotificationType.EMAIL,
destination_value=new_user.email,
verified=False,
enabled=True,
minimum_severity=SeverityLevel.MEDIUM,
)
db.add(destination)
await db.flush()
for company_id in new_company_ids:
db.add(
NotificationDestinationCompany(destination_id=destination.id, company_id=company_id)
)
if pending.make_admin:
new_user.is_admin = True
await PendingProvisioningRepository(db).delete(pending)
return True
+215 -1
View File
@@ -23,7 +23,8 @@ from app.models.enums import EnrichmentStatus, ReportType, SourceStatus
from app.models.report import Report from app.models.report import Report
from app.models.source import Source from app.models.source import Source
from app.models.source_document import SourceDocument from app.models.source_document import SourceDocument
from app.prompts.report_generation import generate_report from app.prompts.report_generation import ReportContent, generate_report
from app.prompts.schemas import ConfidenceLabel, EvidenceRef, Finding
from app.repositories.company_enrichment_repository import CompanyEnrichmentRepository from app.repositories.company_enrichment_repository import CompanyEnrichmentRepository
from app.repositories.report_repository import ReportRepository from app.repositories.report_repository import ReportRepository
from app.services import company_service from app.services import company_service
@@ -31,6 +32,210 @@ from app.services.report_markdown import render_report_markdown
_MAX_DOCUMENTS = 40 _MAX_DOCUMENTS = 40
_MAX_CHANGES = 20 _MAX_CHANGES = 20
_MAX_BACKFILLED_PRODUCTS = 30
_MAX_BACKFILLED_UPDATES = 15
_MAX_BACKFILLED_FUNDING_ROUNDS = 15
def _format_amount(amount: object) -> str | None:
if amount is None:
return None
text = str(amount)
return f"${int(text):,}" if text.isdigit() else text
def _backfill_from_enrichment(content: ReportContent, enrichment: dict | None) -> None:
"""company_enrichment.products/recent_updates/funding map onto
products_and_services/recent_developments/financial_signals with no
inference required - a direct transcription, not a judgment call.
Confirmed live that the LLM is nonetheless inconsistent about
populating these Finding-list fields from enrichment alone (it
reliably uses the same data for prose fields like company_overview,
but repeated real API calls with increasingly explicit instructions
still came back with these lists empty). Rather than keep fighting
prompt compliance for a purely mechanical transform, backfill directly
from the source data whenever the model leaves a field empty despite
the data being available - this only ever *adds* real, non-fabricated
content the model chose not to surface, never overwrites what the
model did produce."""
if not enrichment:
return
if not content.products_and_services:
content.products_and_services = [
Finding(
headline=product["name"],
summary=product.get("description") or product.get("category") or product["name"],
category=product.get("category"),
confidence=ConfidenceLabel.CONFIRMED,
)
for product in enrichment.get("products", [])[:_MAX_BACKFILLED_PRODUCTS]
if product.get("name")
]
if not content.recent_developments:
content.recent_developments = [
Finding(
headline=update["text"],
summary=f"{update.get('type', 'update').capitalize()} published by the company.",
date=update.get("date"),
confidence=ConfidenceLabel.CONFIRMED,
evidence=(
[EvidenceRef(url=update["url"], description="Company-published update.")]
if update.get("url")
else []
),
)
for update in enrichment.get("recent_updates", [])[:_MAX_BACKFILLED_UPDATES]
if update.get("text")
]
if not content.financial_signals:
funding = enrichment.get("funding") or {}
findings = []
total_raised = _format_amount(funding.get("total_raised"))
if total_raised:
findings.append(
Finding(
headline=f"Total funding raised: {total_raised}",
summary="Cumulative funding raised across all rounds, per company_enrichment.",
confidence=ConfidenceLabel.CONFIRMED,
)
)
for round_ in funding.get("rounds", [])[:_MAX_BACKFILLED_FUNDING_ROUNDS]:
name = round_.get("round_name")
if not name:
continue
amount = _format_amount(round_.get("amount"))
investors = round_.get("investors") or []
headline = name.replace("_", " ").title() + (f" - {amount}" if amount else "")
findings.append(
Finding(
headline=headline,
summary=(
f"Investors: {', '.join(investors)}."
if investors
else "No investor detail provided."
),
date=round_.get("date"),
confidence=ConfidenceLabel.CONFIRMED,
)
)
content.financial_signals = findings
def _mirror_swot_into_flat_lists(content: ReportContent) -> None:
"""risks/opportunities are meant to be the same analysis as
swot.threats/swot.opportunities, just in a flat top-level list rather
than nested under swot - not a second, independent judgment call.
Confirmed live: the model reliably populates the SWOT fields but is
inconsistent about also populating these parallel flat fields with the
same content, even though nothing about them requires different
evidence. Mirror rather than re-derive, since the model already did
the real analytical work once."""
if not content.risks and content.swot.threats:
content.risks = list(content.swot.threats)
if not content.opportunities and content.swot.opportunities:
content.opportunities = list(content.swot.opportunities)
def _backfill_reflective_sections(
content: ReportContent,
*,
enrichment: dict | None,
documents: list[dict],
changes: list[dict],
company_name: str,
) -> None:
"""strategic_initiatives, regulatory_and_legal_signals,
unknowns_and_missing_data, and monitoring_recommendations are the
fields the LLM was most persistently reluctant to populate even after
two rounds of explicit prompt strengthening (confirmed live - see
SYSTEM_PROMPT in report_generation.py). Unlike products/recent_updates,
these don't have a single obvious mechanical source, but each still
has SOMETHING honest and non-fabricated to fall back on:
- strategic_initiatives: company_enrichment.specialties names the
company's own stated focus areas - a real, low-confidence signal,
not invented.
- regulatory_and_legal_signals: when truly nothing applies, a single
insufficient_evidence Finding is what the prompt already asks the
model to emit in this situation instead of leaving the list empty -
the model just isn't doing it reliably, so this fills the same gap.
- unknowns_and_missing_data / monitoring_recommendations: these are
meta-analysis of the evidence set itself, not claims about the
company, so they can be derived honestly from what evidence this
pipeline actually did or didn't collect for this company."""
enrichment = enrichment or {}
if not content.strategic_initiatives:
specialties = enrichment.get("specialties") or []
content.strategic_initiatives = [
Finding(
headline=f"Focus area: {specialty}",
summary=(
f"{company_name} lists \"{specialty}\" among its specialties, "
"indicating an area of strategic focus."
),
confidence=ConfidenceLabel.POSSIBLE,
)
for specialty in specialties
if specialty
]
if not content.regulatory_and_legal_signals:
content.regulatory_and_legal_signals = [
Finding(
headline="No regulatory or legal signals identified",
summary=(
"No licensing, compliance, jurisdictional, or legal-structure "
"information was present in the available evidence for "
f"{company_name}."
),
confidence=ConfidenceLabel.INSUFFICIENT_EVIDENCE,
)
]
if not content.unknowns_and_missing_data:
unknowns = []
if not enrichment:
unknowns.append(
"No third-party company enrichment data was available for this company."
)
else:
if not enrichment.get("funding", {}).get("total_raised") and not enrichment.get(
"funding", {}
).get("rounds"):
unknowns.append(
"No detailed financial statements, revenue, or funding figures were found."
)
if not enrichment.get("leadership_team"):
unknowns.append("No leadership or executive team data was found.")
if not enrichment.get("customers"):
unknowns.append("No named customers or case studies were found.")
if not documents:
unknowns.append(
"No source documents have been collected yet from ongoing monitoring, so "
"hiring, technology, patent, and manufacturing signals are not yet available."
)
if not changes:
unknowns.append(
"No changes have been detected yet between monitoring runs for this company."
)
content.unknowns_and_missing_data = unknowns
if not content.monitoring_recommendations:
recommendations = [
f"Monitor company_enrichment for {company_name} on its next refresh for changes "
"to products, leadership, or funding.",
"Track newly published company updates for announcements of new initiatives "
"or partnerships.",
]
if documents:
recommendations.append(
"Continue reviewing newly collected source documents for signals not yet "
"reflected in company_enrichment."
)
content.monitoring_recommendations = recommendations
async def _gather_documents(db: AsyncSession, company_id: uuid.UUID) -> list[dict]: async def _gather_documents(db: AsyncSession, company_id: uuid.UUID) -> list[dict]:
@@ -130,6 +335,15 @@ async def generate_and_persist_report(
public_identifiers=company.public_identifiers, public_identifiers=company.public_identifiers,
enrichment=enrichment, enrichment=enrichment,
) )
_backfill_from_enrichment(content, enrichment)
_mirror_swot_into_flat_lists(content)
_backfill_reflective_sections(
content,
enrichment=enrichment,
documents=documents,
changes=changes,
company_name=company.name,
)
generated_at = datetime.now(UTC) generated_at = datetime.now(UTC)
model_name = _model_name(settings, llm) model_name = _model_name(settings, llm)
@@ -0,0 +1,45 @@
"""add pending_provisionings table
Revision ID: 2c2a563a5edb
Revises: 9e6c80c11da7
Create Date: 2026-08-06 22:22:22.348339
"""
from __future__ import annotations
from collections.abc import Sequence
from alembic import op
import sqlalchemy as sa
revision: str = '2c2a563a5edb'
down_revision: str | None = '9e6c80c11da7'
branch_labels: Sequence[str] | str | None = None
depends_on: Sequence[str] | str | None = None
def upgrade() -> None:
# Autogenerate also proposed adding a redundant UNIQUE constraint on the
# `id` column of every other existing table (a cosmetic artifact of how
# this project's earlier migrations declared unnamed unique constraints,
# unrelated to this change) - stripped out, keeping only the new table.
op.create_table('pending_provisionings',
sa.Column('email', sa.String(length=320), nullable=False),
sa.Column('source_user_id', sa.Uuid(), nullable=False),
sa.Column('make_admin', sa.Boolean(), nullable=False),
sa.Column('id', sa.Uuid(), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), nullable=False),
sa.Column('updated_at', sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(['source_user_id'], ['users.id'], ondelete='CASCADE'),
sa.PrimaryKeyConstraint('id'),
sa.UniqueConstraint('id')
)
op.create_index(op.f('ix_pending_provisionings_email'), 'pending_provisionings', ['email'], unique=True)
op.create_index(op.f('ix_pending_provisionings_source_user_id'), 'pending_provisionings', ['source_user_id'], unique=False)
def downgrade() -> None:
op.drop_index(op.f('ix_pending_provisionings_source_user_id'), table_name='pending_provisionings')
op.drop_index(op.f('ix_pending_provisionings_email'), table_name='pending_provisionings')
op.drop_table('pending_provisionings')
@@ -0,0 +1,32 @@
"""add must_change_password to users
Revision ID: 9e6c80c11da7
Revises: c34c769afc07
Create Date: 2026-08-06 03:10:00.000000
"""
from __future__ import annotations
from collections.abc import Sequence
import sqlalchemy as sa
from alembic import op
revision: str = "9e6c80c11da7"
down_revision: str | None = "c34c769afc07"
branch_labels: Sequence[str] | str | None = None
depends_on: Sequence[str] | str | None = None
def upgrade() -> None:
op.add_column(
"users",
sa.Column(
"must_change_password", sa.Boolean(), nullable=False, server_default=sa.false()
),
)
def downgrade() -> None:
op.drop_column("users", "must_change_password")
+25
View File
@@ -48,6 +48,31 @@ def _setup_database():
async with engine.begin() as conn: async with engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all) await conn.run_sync(Base.metadata.create_all)
# auth_service.register() auto-promotes the first-ever registration
# to admin (see its docstring) so a fresh deployment always has a
# bootstrap path to admin access. Without this seed row, whichever
# test happens to register a "plain" user first in this shared
# session-wide DB would non-deterministically become an admin,
# breaking every non-admin-should-get-403 test depending on
# execution order. Seeding one admin up front keeps that dedicated
# to test_first_registered_user_becomes_admin (which starts from a
# genuinely empty users table), while every other test's "plain"
# registration behaves exactly as before.
from app.models.user import User
async with get_sessionmaker()() as session:
session.add(
User(
email="[email protected]",
password_hash=None,
display_name="Seed Admin",
timezone="UTC",
is_admin=True,
email_verified=True,
)
)
await session.commit()
asyncio.run(_create()) asyncio.run(_create())
yield yield
@@ -0,0 +1,260 @@
"""Account deletion (DELETE /auth/me) and admin-forced password change
(POST /admin/users/require-password-change -> POST /auth/change-password,
enforced by app.auth.dependencies.get_current_user)."""
from __future__ import annotations
import uuid
from fastapi.testclient import TestClient
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.enums import SecurityEventType
from app.repositories.user_repository import UserRepository
from app.repositories.user_security_event_repository import UserSecurityEventRepository
def _unique_email() -> str:
return f"user-{uuid.uuid4().hex[:12]}@example.com"
def _register_and_login(client: TestClient, password: str = "correct-horse-1") -> dict:
email = _unique_email()
client.post(
"/api/v1/auth/register",
json={"email": email, "password": password, "display_name": "T"},
)
tokens = client.post("/api/v1/auth/login", json={"email": email, "password": password}).json()
return {"email": email, "headers": {"Authorization": f"Bearer {tokens['access_token']}"}}
async def _register_admin_and_login(client: TestClient, db_session: AsyncSession) -> dict[str, str]:
email = f"admin-{uuid.uuid4().hex[:12]}@example.com"
client.post(
"/api/v1/auth/register",
json={"email": email, "password": "correct-horse-1", "display_name": "Admin"},
)
user = await UserRepository(db_session).get_by_email(email)
user.is_admin = True
await db_session.commit()
tokens = client.post(
"/api/v1/auth/login", json={"email": email, "password": "correct-horse-1"}
).json()
return {"Authorization": f"Bearer {tokens['access_token']}"}
# --- DELETE /auth/me -------------------------------------------------------
def test_delete_account_requires_auth(client: TestClient):
resp = client.request(
"DELETE", "/api/v1/auth/me", json={"password": "whatever"}
)
assert resp.status_code == 401
def test_delete_account_rejects_wrong_password(client: TestClient):
account = _register_and_login(client)
resp = client.request(
"DELETE",
"/api/v1/auth/me",
json={"password": "not-the-right-password"},
headers=account["headers"],
)
assert resp.status_code == 401
def test_delete_account_succeeds_with_correct_password_and_logs_out_the_account(
client: TestClient,
):
account = _register_and_login(client)
resp = client.request(
"DELETE",
"/api/v1/auth/me",
json={"password": "correct-horse-1"},
headers=account["headers"],
)
assert resp.status_code == 204
# The account is gone - the same token no longer resolves to anyone.
me_resp = client.get("/api/v1/auth/me", headers=account["headers"])
assert me_resp.status_code == 401
# And a fresh login attempt with the same credentials fails too.
login_resp = client.post(
"/api/v1/auth/login",
json={"email": account["email"], "password": "correct-horse-1"},
)
assert login_resp.status_code == 401
async def test_delete_account_cascades_to_owned_data(client: TestClient, db_session: AsyncSession):
"""A company created by the account must be gone too (ON DELETE CASCADE),
not just the user row."""
account = _register_and_login(client)
create_resp = client.post(
"/api/v1/companies",
json={"name": f"Co-{uuid.uuid4().hex[:8]}", "official_website": None},
headers=account["headers"],
)
assert create_resp.status_code == 201
company_id = create_resp.json()["id"]
client.request(
"DELETE", "/api/v1/auth/me", json={"password": "correct-horse-1"}, headers=account["headers"]
)
from sqlalchemy import text
row = await db_session.execute(
text("SELECT 1 FROM companies WHERE id = :id"), {"id": company_id}
)
assert row.first() is None
# --- POST /admin/users/require-password-change -----------------------------
def test_require_password_change_requires_admin(client: TestClient):
account = _register_and_login(client)
resp = client.post(
"/api/v1/admin/users/require-password-change",
json={"email": account["email"]},
headers=account["headers"],
)
assert resp.status_code == 403
async def test_require_password_change_rejects_unknown_email(
client: TestClient, db_session: AsyncSession
):
headers = await _register_admin_and_login(client, db_session)
resp = client.post(
"/api/v1/admin/users/require-password-change",
json={"email": "[email protected]"},
headers=headers,
)
assert resp.status_code == 404
async def test_admin_flags_an_account_and_it_gets_logged_to_the_targets_activity(
client: TestClient, db_session: AsyncSession
):
"""Checked directly against the DB, not via GET /auth/security-events -
that endpoint isn't in the must-change-password exempt list, so the
just-flagged account can't reach it until they change their password
(see test_flagged_account_is_blocked_from_other_endpoints)."""
admin_headers = await _register_admin_and_login(client, db_session)
account = _register_and_login(client)
resp = client.post(
"/api/v1/admin/users/require-password-change",
json={"email": account["email"]},
headers=admin_headers,
)
assert resp.status_code == 204
user = await UserRepository(db_session).get_by_email(account["email"])
assert user.must_change_password is True
events = await UserSecurityEventRepository(db_session).list_for_user(user.id)
assert any(e.event_type == SecurityEventType.PASSWORD_CHANGE_REQUIRED for e in events)
# --- Enforcement gate: everything except me/change-password/logout 403s ----
async def test_flagged_account_is_blocked_from_other_endpoints(
client: TestClient, db_session: AsyncSession
):
admin_headers = await _register_admin_and_login(client, db_session)
account = _register_and_login(client)
client.post(
"/api/v1/admin/users/require-password-change",
json={"email": account["email"]},
headers=admin_headers,
)
blocked = client.get("/api/v1/companies", headers=account["headers"])
assert blocked.status_code == 403
still_ok = client.get("/api/v1/auth/me", headers=account["headers"])
assert still_ok.status_code == 200
assert still_ok.json()["must_change_password"] is True
async def test_change_password_clears_the_flag_and_unblocks_the_account(
client: TestClient, db_session: AsyncSession
):
admin_headers = await _register_admin_and_login(client, db_session)
account = _register_and_login(client)
client.post(
"/api/v1/admin/users/require-password-change",
json={"email": account["email"]},
headers=admin_headers,
)
change_resp = client.post(
"/api/v1/auth/change-password",
json={"current_password": "correct-horse-1", "new_password": "brand-new-horse-2"},
headers=account["headers"],
)
assert change_resp.status_code == 200
new_tokens = change_resp.json()
new_headers = {"Authorization": f"Bearer {new_tokens['access_token']}"}
me_resp = client.get("/api/v1/auth/me", headers=new_headers)
assert me_resp.status_code == 200
assert me_resp.json()["must_change_password"] is False
unblocked = client.get("/api/v1/companies", headers=new_headers)
assert unblocked.status_code == 200
# The new password actually works on a fresh login.
login_resp = client.post(
"/api/v1/auth/login",
json={"email": account["email"], "password": "brand-new-horse-2"},
)
assert login_resp.status_code == 200
def test_change_password_rejects_wrong_current_password(client: TestClient):
account = _register_and_login(client)
resp = client.post(
"/api/v1/auth/change-password",
json={"current_password": "totally-wrong", "new_password": "brand-new-horse-2"},
headers=account["headers"],
)
assert resp.status_code == 401
def test_change_password_rejects_reusing_the_current_password(client: TestClient):
account = _register_and_login(client)
resp = client.post(
"/api/v1/auth/change-password",
json={"current_password": "correct-horse-1", "new_password": "correct-horse-1"},
headers=account["headers"],
)
assert resp.status_code == 400
def test_change_password_revokes_the_old_refresh_token(client: TestClient):
email = _unique_email()
client.post(
"/api/v1/auth/register",
json={"email": email, "password": "correct-horse-1", "display_name": "T"},
)
tokens = client.post(
"/api/v1/auth/login", json={"email": email, "password": "correct-horse-1"}
).json()
headers = {"Authorization": f"Bearer {tokens['access_token']}"}
client.post(
"/api/v1/auth/change-password",
json={"current_password": "correct-horse-1", "new_password": "brand-new-horse-2"},
headers=headers,
)
stale_refresh_resp = client.post(
"/api/v1/auth/refresh", json={"refresh_token": tokens["refresh_token"]}
)
assert stale_refresh_resp.status_code == 401
+34
View File
@@ -63,6 +63,40 @@ def test_register_rejects_weak_password(client):
assert resp.status_code == 422 assert resp.status_code == 422
async def test_first_registration_on_an_admin_less_deployment_becomes_admin(db_session, settings):
"""Bootstraps admin access on a fresh deployment - without this, the
only way to ever get an admin account is direct DB access. Patches
count_admins() to simulate a genuinely admin-less deployment rather
than manipulating the shared session-wide test DB's real admin count
(conftest.py seeds one admin precisely so ordinary registrations in
other tests never accidentally trip this path)."""
from unittest.mock import patch
from app.repositories.user_repository import UserRepository
from app.schemas.auth import RegisterRequest
from app.services.auth_service import register
payload = RegisterRequest(
email=_unique_email(), password="correct-horse-1", display_name="First User"
)
with patch.object(UserRepository, "count_admins", return_value=0):
user = await register(db_session, settings, "127.0.0.1", payload)
assert user.is_admin is True
async def test_registration_after_an_admin_already_exists_is_not_promoted(db_session, settings):
from app.schemas.auth import RegisterRequest
from app.services.auth_service import register
payload = RegisterRequest(
email=_unique_email(), password="correct-horse-1", display_name="Second User"
)
user = await register(db_session, settings, "127.0.0.1", payload)
assert user.is_admin is False
def test_login_wrong_password_rejected(client): def test_login_wrong_password_rejected(client):
email = _unique_email() email = _unique_email()
client.post( client.post(
+73
View File
@@ -3,6 +3,9 @@
from __future__ import annotations from __future__ import annotations
import uuid import uuid
from unittest.mock import patch
import pytest
def _register_and_login(client) -> dict[str, str]: def _register_and_login(client) -> dict[str, str]:
@@ -252,3 +255,73 @@ def test_create_company_does_not_enqueue_enrichment_without_a_key(client):
assert resp.status_code == 201 assert resp.status_code == 201
mock_delay.assert_not_called() mock_delay.assert_not_called()
assert resp.json()["enrichment"] is None assert resp.json()["enrichment"] is None
def test_refresh_enrichment_section_requires_authentication(client):
resp = client.post(f"/api/v1/companies/{uuid.uuid4()}/enrichment/sections/products/refresh")
assert resp.status_code == 401
def test_refresh_enrichment_section_rejects_an_unknown_section(client):
headers = _register_and_login(client)
company = _create_company(client, headers).json()
resp = client.post(
f"/api/v1/companies/{company['id']}/enrichment/sections/not_a_section/refresh",
headers=headers,
)
assert resp.status_code == 422
def test_refresh_enrichment_section_404s_without_an_enrichment_record(client):
headers = _register_and_login(client)
company = _create_company(client, headers).json() # no NINJAPEAR_API_KEY -> no record
resp = client.post(
f"/api/v1/companies/{company['id']}/enrichment/sections/products/refresh",
headers=headers,
)
assert resp.status_code == 404
def test_refresh_enrichment_section_is_scoped_to_owner(client):
owner_headers = _register_and_login(client)
other_headers = _register_and_login(client)
company = _create_company(client, owner_headers).json()
resp = client.post(
f"/api/v1/companies/{company['id']}/enrichment/sections/products/refresh",
headers=other_headers,
)
assert resp.status_code == 404
@pytest.mark.asyncio
async def test_refresh_enrichment_section_updates_just_that_section(client, db_session, settings):
from app.enrichment.base import Product
from app.enrichment.mock import MockEnrichmentProvider
from app.repositories.company_repository import CompanyRepository
from app.services.enrichment_service import enrich_company
headers = _register_and_login(client)
company_id = _create_company(client, headers).json()["id"]
company = await CompanyRepository(db_session).get_by_id(uuid.UUID(company_id))
await enrich_company(db_session, settings, MockEnrichmentProvider(), company)
class _ProductsProvider(MockEnrichmentProvider):
async def get_products(self, name, website):
return [Product(name="Acme Pay", description="Payments product")]
with patch("app.api.v1.companies.get_enrichment_provider", return_value=_ProductsProvider()):
resp = client.post(
f"/api/v1/companies/{company_id}/enrichment/sections/products/refresh",
headers=headers,
)
assert resp.status_code == 200
products = resp.json()["enrichment"]["data"]["products"]
assert products == [{"name": "Acme Pay", "description": "Payments product", "category": None}]
+219
View File
@@ -0,0 +1,219 @@
"""Settings -> Database viewer: mint (admin-gated) -> bootstrap (token ->
session cookie) -> verify (cookie -> live is_admin check) round trip.
Cookies are extracted from Set-Cookie headers and passed explicitly on
follow-up requests rather than relying on TestClient's cookie jar - the
session cookie is marked Secure, and TestClient's base_url is plain http,
so a real cookie jar wouldn't resend it anyway."""
from __future__ import annotations
import uuid
from datetime import UTC, datetime, timedelta
import jwt
from fastapi.testclient import TestClient
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.config import get_settings
from app.repositories.user_repository import UserRepository
def _unique_email() -> str:
return f"user-{uuid.uuid4().hex[:12]}@example.com"
async def _register_admin_and_login(client: TestClient, db_session: AsyncSession) -> dict[str, str]:
email = f"admin-{uuid.uuid4().hex[:12]}@example.com"
client.post(
"/api/v1/auth/register",
json={"email": email, "password": "correct-horse-1", "display_name": "Admin"},
)
user = await UserRepository(db_session).get_by_email(email)
user.is_admin = True
await db_session.commit()
tokens = client.post(
"/api/v1/auth/login", json={"email": email, "password": "correct-horse-1"}
).json()
return {"Authorization": f"Bearer {tokens['access_token']}"}
async def _register_non_admin_and_login(client: TestClient) -> dict[str, str]:
email = _unique_email()
client.post(
"/api/v1/auth/register",
json={"email": email, "password": "correct-horse-1", "display_name": "T"},
)
tokens = client.post(
"/api/v1/auth/login", json={"email": email, "password": "correct-horse-1"}
).json()
return {"Authorization": f"Bearer {tokens['access_token']}"}
def _extract_cookie(set_cookie_header: str) -> tuple[str, str]:
first_pair = set_cookie_header.split(";", 1)[0]
name, value = first_pair.split("=", 1)
return name, value
# --- /db-viewer/session (mint) --------------------------------------------
def test_create_session_requires_auth(client: TestClient):
resp = client.post("/api/v1/db-viewer/session")
assert resp.status_code == 401
async def test_create_session_non_admin_forbidden(client: TestClient):
headers = await _register_non_admin_and_login(client)
resp = client.post("/api/v1/db-viewer/session", headers=headers)
assert resp.status_code == 403
async def test_admin_can_mint_a_bootstrap_token(client: TestClient, db_session: AsyncSession):
headers = await _register_admin_and_login(client, db_session)
resp = client.post("/api/v1/db-viewer/session", headers=headers)
assert resp.status_code == 200
assert resp.json()["token"]
async def test_minting_a_session_is_logged_to_the_admins_account_activity(
client: TestClient, db_session: AsyncSession
):
headers = await _register_admin_and_login(client, db_session)
client.post("/api/v1/db-viewer/session", headers=headers)
events = client.get("/api/v1/auth/security-events", headers=headers).json()
assert any(e["event_type"] == "db_viewer_session_created" for e in events)
# --- /db-viewer/bootstrap (token -> cookie) -------------------------------
async def test_bootstrap_with_a_valid_token_sets_a_session_cookie_and_redirects(
client: TestClient, db_session: AsyncSession
):
headers = await _register_admin_and_login(client, db_session)
token = client.post("/api/v1/db-viewer/session", headers=headers).json()["token"]
resp = client.get(
"/api/v1/db-viewer/bootstrap", params={"token": token}, follow_redirects=False
)
assert resp.status_code == 302
assert resp.headers["location"] == "/?pgsql=postgres"
assert "db_viewer_session" in resp.headers["set-cookie"]
assert "HttpOnly" in resp.headers["set-cookie"]
assert "Secure" in resp.headers["set-cookie"]
async def test_bootstrap_success_is_logged_to_the_admins_account_activity(
client: TestClient, db_session: AsyncSession
):
headers = await _register_admin_and_login(client, db_session)
token = client.post("/api/v1/db-viewer/session", headers=headers).json()["token"]
client.get("/api/v1/db-viewer/bootstrap", params={"token": token}, follow_redirects=False)
events = client.get("/api/v1/auth/security-events", headers=headers).json()
assert any(e["event_type"] == "db_viewer_accessed" for e in events)
def test_bootstrap_rejects_a_garbage_token(client: TestClient):
resp = client.get(
"/api/v1/db-viewer/bootstrap", params={"token": "not-a-real-token"}, follow_redirects=False
)
assert resp.status_code == 403
def test_bootstrap_rejects_an_expired_token(client: TestClient):
settings = get_settings()
now = datetime.now(UTC)
expired_token = jwt.encode(
{
"sub": str(uuid.uuid4()),
"type": "db_viewer_bootstrap",
"iat": now - timedelta(minutes=10),
"exp": now - timedelta(minutes=5),
"jti": "x",
},
settings.jwt_secret,
algorithm="HS256",
)
resp = client.get(
"/api/v1/db-viewer/bootstrap", params={"token": expired_token}, follow_redirects=False
)
assert resp.status_code == 403
def test_bootstrap_rejects_a_session_type_token_used_as_a_bootstrap_token(client: TestClient):
"""Type confusion guard: a DB_VIEWER_SESSION token must not work as a
DB_VIEWER_BOOTSTRAP token, even though both are signed with the same
jwt_secret."""
settings = get_settings()
now = datetime.now(UTC)
session_typed_token = jwt.encode(
{
"sub": str(uuid.uuid4()),
"type": "db_viewer_session",
"iat": now,
"exp": now + timedelta(minutes=2),
"jti": "x",
},
settings.jwt_secret,
algorithm="HS256",
)
resp = client.get(
"/api/v1/db-viewer/bootstrap",
params={"token": session_typed_token},
follow_redirects=False,
)
assert resp.status_code == 403
# --- /db-viewer/verify (cookie -> live is_admin check) --------------------
async def test_verify_succeeds_for_a_live_admin_with_a_valid_session_cookie(
client: TestClient, db_session: AsyncSession
):
headers = await _register_admin_and_login(client, db_session)
token = client.post("/api/v1/db-viewer/session", headers=headers).json()["token"]
bootstrap_resp = client.get(
"/api/v1/db-viewer/bootstrap", params={"token": token}, follow_redirects=False
)
cookie_name, cookie_value = _extract_cookie(bootstrap_resp.headers["set-cookie"])
verify_resp = client.get("/api/v1/db-viewer/verify", cookies={cookie_name: cookie_value})
assert verify_resp.status_code == 200
def test_verify_rejects_a_missing_cookie(client: TestClient):
resp = client.get("/api/v1/db-viewer/verify")
assert resp.status_code == 401
async def test_verify_rejects_a_session_cookie_once_admin_is_revoked(
client: TestClient, db_session: AsyncSession
):
headers = await _register_admin_and_login(client, db_session)
token = client.post("/api/v1/db-viewer/session", headers=headers).json()["token"]
bootstrap_resp = client.get(
"/api/v1/db-viewer/bootstrap", params={"token": token}, follow_redirects=False
)
cookie_name, cookie_value = _extract_cookie(bootstrap_resp.headers["set-cookie"])
# Sanity: still works before revocation.
assert (
client.get("/api/v1/db-viewer/verify", cookies={cookie_name: cookie_value}).status_code
== 200
)
admin_email_resp = client.get("/api/v1/auth/me", headers=headers)
admin = await UserRepository(db_session).get_by_email(admin_email_resp.json()["email"])
admin.is_admin = False
await db_session.commit()
assert (
client.get("/api/v1/db-viewer/verify", cookies={cookie_name: cookie_value}).status_code
== 401
)
+99 -3
View File
@@ -7,24 +7,31 @@ import uuid
import pytest import pytest
from app.core.errors import NotFoundError
from app.enrichment.base import ( from app.enrichment.base import (
CompanyDetails, CompanyDetails,
CompanyFunding, CompanyFunding,
LeadershipMember, LeadershipMember,
Product,
) )
from app.models.company import Company from app.models.company import Company
from app.models.enums import EnrichmentStatus from app.models.enums import EnrichmentStatus
from app.repositories.company_enrichment_repository import CompanyEnrichmentRepository from app.repositories.company_enrichment_repository import CompanyEnrichmentRepository
from app.services.enrichment_service import enrich_company from app.services.enrichment_service import enrich_company, refresh_section
class _FakeProvider: class _FakeProvider:
provider_name = "fake" provider_name = "fake"
def __init__( def __init__(
self, *, leadership: list[LeadershipMember] | None = None, fail: set[str] | None = None self,
*,
leadership: list[LeadershipMember] | None = None,
products: list[Product] | None = None,
fail: set[str] | None = None,
): ):
self._leadership = leadership or [] self._leadership = leadership or []
self._products = products if products is not None else []
self._fail = fail or set() self._fail = fail or set()
async def get_company_details(self, name, website): async def get_company_details(self, name, website):
@@ -50,7 +57,7 @@ class _FakeProvider:
async def get_products(self, name, website): async def get_products(self, name, website):
if "products" in self._fail: if "products" in self._fail:
raise RuntimeError("products boom") raise RuntimeError("products boom")
return [] return self._products
async def get_customers(self, name, website): async def get_customers(self, name, website):
if "customers" in self._fail: if "customers" in self._fail:
@@ -168,3 +175,92 @@ async def test_upsert_updates_the_same_row_on_a_second_call(db_session, settings
stored = await CompanyEnrichmentRepository(db_session).get_for_company(company.id) stored = await CompanyEnrichmentRepository(db_session).get_for_company(company.id)
assert stored.id == first.id assert stored.id == first.id
assert stored.status == EnrichmentStatus.PARTIAL assert stored.status == EnrichmentStatus.PARTIAL
@pytest.mark.asyncio
async def test_refresh_section_replaces_only_that_section(db_session, settings):
company = await _make_company(db_session)
await enrich_company(db_session, settings, _FakeProvider(), company) # products come back []
provider = _FakeProvider(products=[Product(name="Acme Pay", description="Payments product")])
enrichment = await refresh_section(db_session, settings, provider, company, "products")
assert [p["name"] for p in enrichment.data["products"]] == ["Acme Pay"]
assert enrichment.data["description"] == "A company." # untouched by the refresh
assert enrichment.status == EnrichmentStatus.COMPLETE
@pytest.mark.asyncio
async def test_refresh_section_records_a_fresh_failure_without_losing_other_data(
db_session, settings
):
company = await _make_company(db_session)
await enrich_company(db_session, settings, _FakeProvider(), company)
failing_provider = _FakeProvider(fail={"products"})
enrichment = await refresh_section(db_session, settings, failing_provider, company, "products")
assert enrichment.errors["products"] == "products boom"
assert enrichment.data["description"] == "A company."
assert enrichment.status == EnrichmentStatus.PARTIAL
@pytest.mark.asyncio
async def test_refresh_section_clears_a_previously_recorded_error(db_session, settings):
company = await _make_company(db_session)
await enrich_company(db_session, settings, _FakeProvider(fail={"products"}), company)
provider = _FakeProvider(products=[Product(name="Acme Pay")])
enrichment = await refresh_section(db_session, settings, provider, company, "products")
assert "products" not in enrichment.errors
assert enrichment.status == EnrichmentStatus.COMPLETE
@pytest.mark.asyncio
async def test_refresh_section_preserves_per_leader_lookups_on_a_details_refresh(
db_session, settings
):
company = await _make_company(db_session)
leadership = [LeadershipMember(name="Jane Doe")]
await enrich_company(db_session, settings, _FakeProvider(leadership=leadership), company)
stored = await CompanyEnrichmentRepository(db_session).get_for_company(company.id)
assert stored.data["leadership_team"][0]["work_email"] == "[email protected]"
# A details refresh re-fetches the roster but not the per-leader lookups -
# the previously-found work_email must survive, not silently disappear.
provider = _FakeProvider(leadership=[LeadershipMember(name="Jane Doe", title="CEO")])
enrichment = await refresh_section(db_session, settings, provider, company, "details")
member = enrichment.data["leadership_team"][0]
assert member["title"] == "CEO"
assert member["work_email"] == "[email protected]"
@pytest.mark.asyncio
async def test_refresh_section_rejects_an_unknown_section(db_session, settings):
company = await _make_company(db_session)
await enrich_company(db_session, settings, _FakeProvider(), company)
with pytest.raises(ValueError):
await refresh_section(db_session, settings, _FakeProvider(), company, "not_a_section")
@pytest.mark.asyncio
async def test_refresh_section_requires_an_existing_enrichment_record(db_session, settings):
company = await _make_company(db_session) # enrichment was never run
with pytest.raises(NotFoundError):
await refresh_section(db_session, settings, _FakeProvider(), company, "products")
@pytest.mark.asyncio
async def test_refresh_section_adds_its_own_credit_cost(db_session, settings):
company = await _make_company(db_session)
baseline = await enrich_company(db_session, settings, _FakeProvider(), company)
baseline_credits = baseline.credits_spent # `upsert` mutates rows in place - snapshot first
enrichment = await refresh_section(db_session, settings, _FakeProvider(), company, "products")
assert enrichment.credits_spent == baseline_credits + 3 # products' own credit cost
+429
View File
@@ -0,0 +1,429 @@
"""Pending account provisioning: an admin queues up an email in advance,
and the moment that email actually verifies a real account, it gets a
deep copy of another user's API keys and companies (every related table,
not just the company row itself), a notification destination for its own
address, and optionally an admin promotion. See provisioning_service.py."""
from __future__ import annotations
import asyncio
import uuid
from datetime import UTC, datetime, timedelta
import pytest
from sqlalchemy import select
from app.core.errors import ConflictError, NotFoundError
from app.core.security import hash_email_code, hash_password
from app.db.session import get_sessionmaker
from app.models.company import Company, CompanyAlias, Competitor
from app.models.company_enrichment import CompanyEnrichment
from app.models.detected_change import DetectedChange
from app.models.email_code import EmailCode
from app.models.enums import (
ApiKeyProvider,
ChangeStatus,
ChangeType,
CompanyStatus,
EmailCodePurpose,
EnrichmentStatus,
MonitoringFrequency,
MonitoringRunStatus,
MonitoringRunTrigger,
ReportType,
SeverityLevel,
SourceStatus,
SourceType,
)
from app.models.monitor_configuration import MonitorConfiguration
from app.models.monitoring_run import MonitoringRun
from app.models.report import Report
from app.models.snapshot import Snapshot
from app.models.source import Source
from app.models.source_document import SourceDocument
from app.models.user import User
from app.models.user_api_key import UserApiKey
from app.repositories.notification_destination_repository import (
NotificationDestinationRepository,
)
from app.repositories.pending_provisioning_repository import PendingProvisioningRepository
from app.repositories.user_repository import UserRepository
from app.services import provisioning_service
async def _make_user(db_session, *, email: str | None = None, is_admin: bool = False) -> User:
user = User(
id=uuid.uuid4(),
email=(email or f"user-{uuid.uuid4().hex[:10]}@example.com").lower(),
password_hash=hash_password("correct-horse-1"),
display_name="Test User",
timezone="UTC",
is_admin=is_admin,
email_verified=True,
)
db_session.add(user)
await db_session.flush()
return user
async def _build_full_company(db_session, owner: User) -> Company:
"""One company with at least one row in every table _clone_company
touches, so the deep-clone test actually exercises every FK remap."""
company = Company(
user_id=owner.id,
name="Acme Corp",
slug=f"acme-{uuid.uuid4().hex[:6]}",
official_website="https://acme.example.com",
status=CompanyStatus.ACTIVE,
)
db_session.add(company)
await db_session.flush()
db_session.add(CompanyAlias(company_id=company.id, alias="Acme"))
db_session.add(Competitor(company_id=company.id, name="Rival Co"))
db_session.add(
MonitorConfiguration(
company_id=company.id, frequency_type=MonitoringFrequency.WEEKLY, enabled=True
)
)
db_session.add(
CompanyEnrichment(
company_id=company.id,
status=EnrichmentStatus.COMPLETE,
data={"products": [{"name": "Widget"}]},
errors={},
credits_spent=5,
)
)
source = Source(
company_id=company.id,
source_type=SourceType.WEBSITE,
name="Homepage",
status=SourceStatus.ACTIVE,
)
db_session.add(source)
await db_session.flush()
db_session.add(
SourceDocument(
source_id=source.id,
company_id=company.id,
url="https://acme.example.com/page",
canonical_url="https://acme.example.com/page",
title="Homepage",
content_text="Hello world",
content_hash="abc123",
retrieved_date=datetime.now(UTC),
extraction_method="static_html",
)
)
run = MonitoringRun(
company_id=company.id,
trigger_type=MonitoringRunTrigger.MANUAL,
status=MonitoringRunStatus.SUCCESSFUL,
)
db_session.add(run)
await db_session.flush()
db_session.add(
Report(
company_id=company.id,
monitoring_run_id=run.id,
report_type=ReportType.MANUAL,
title="Acme Report",
executive_summary="Summary",
structured_report={"executive_summary": "Summary"},
markdown_content="# Acme Report",
model_provider="mock",
model_name="mock",
)
)
snapshot = Snapshot(
company_id=company.id,
source_id=source.id,
monitoring_run_id=run.id,
snapshot_type="structured",
hash="snaphash",
)
db_session.add(snapshot)
await db_session.flush()
db_session.add(
DetectedChange(
company_id=company.id,
source_id=source.id,
monitoring_run_id=run.id,
previous_snapshot_id=snapshot.id,
current_snapshot_id=snapshot.id,
change_type=ChangeType.CONTENT_MODIFIED,
severity=SeverityLevel.LOW,
status=ChangeStatus.NEW,
summary="Something changed",
confidence_score=0.5,
significance_score=0.5,
)
)
db_session.add(
UserApiKey(
user_id=owner.id, provider=ApiKeyProvider.ANTHROPIC, encrypted_key="encrypted-blob"
)
)
await db_session.commit()
return company
@pytest.mark.asyncio
async def test_create_pending_rejects_an_email_that_already_has_an_account(db_session, settings):
owner = await _make_user(db_session)
existing = await _make_user(db_session, email="[email protected]")
with pytest.raises(ConflictError):
await provisioning_service.create_pending(db_session, existing.email, owner.id, False)
@pytest.mark.asyncio
async def test_create_pending_rejects_a_duplicate_pending_email(db_session, settings):
owner = await _make_user(db_session)
email = f"pending-{uuid.uuid4().hex[:10]}@example.com"
await provisioning_service.create_pending(db_session, email, owner.id, False)
with pytest.raises(ConflictError):
await provisioning_service.create_pending(db_session, email, owner.id, False)
@pytest.mark.asyncio
async def test_apply_if_pending_is_a_no_op_without_a_matching_entry(db_session, settings):
new_user = await _make_user(db_session, email="[email protected]")
applied = await provisioning_service.apply_if_pending(db_session, new_user)
assert applied is False
@pytest.mark.asyncio
async def test_apply_if_pending_deep_clones_everything_and_is_consumed_once(db_session, settings):
owner = await _make_user(db_session)
company = await _build_full_company(db_session, owner)
email = f"invitee-{uuid.uuid4().hex[:10]}@example.com"
await provisioning_service.create_pending(db_session, email, owner.id, make_admin=True)
new_user = await _make_user(db_session, email=email)
applied = await provisioning_service.apply_if_pending(db_session, new_user)
await db_session.commit()
assert applied is True
assert new_user.is_admin is True
# The pending entry only ever fires once.
assert await PendingProvisioningRepository(db_session).get_by_email(email) is None
applied_again = await provisioning_service.apply_if_pending(db_session, new_user)
assert applied_again is False
new_company = (
await db_session.execute(select(Company).where(Company.user_id == new_user.id))
).scalar_one()
assert new_company.id != company.id
assert new_company.name == "Acme Corp"
assert (
await db_session.execute(
select(CompanyAlias).where(CompanyAlias.company_id == new_company.id)
)
).scalar_one().alias == "Acme"
assert (
await db_session.execute(select(Competitor).where(Competitor.company_id == new_company.id))
).scalar_one().name == "Rival Co"
assert (
(
await db_session.execute(
select(MonitorConfiguration).where(
MonitorConfiguration.company_id == new_company.id
)
)
)
.scalar_one()
.enabled
)
new_source = (
await db_session.execute(select(Source).where(Source.company_id == new_company.id))
).scalar_one()
assert (
new_source.id
!= (await db_session.execute(select(Source).where(Source.company_id == company.id)))
.scalar_one()
.id
)
new_doc = (
await db_session.execute(
select(SourceDocument).where(SourceDocument.company_id == new_company.id)
)
).scalar_one()
assert new_doc.source_id == new_source.id # remapped to the CLONED source, not the original
new_run = (
await db_session.execute(
select(MonitoringRun).where(MonitoringRun.company_id == new_company.id)
)
).scalar_one()
new_report = (
await db_session.execute(select(Report).where(Report.company_id == new_company.id))
).scalar_one()
assert new_report.monitoring_run_id == new_run.id
new_snapshot = (
await db_session.execute(select(Snapshot).where(Snapshot.company_id == new_company.id))
).scalar_one()
assert new_snapshot.source_id == new_source.id
assert new_snapshot.monitoring_run_id == new_run.id
new_change = (
await db_session.execute(
select(DetectedChange).where(DetectedChange.company_id == new_company.id)
)
).scalar_one()
assert new_change.previous_snapshot_id == new_snapshot.id
assert new_change.current_snapshot_id == new_snapshot.id
assert new_change.source_id == new_source.id
assert new_change.monitoring_run_id == new_run.id
new_enrichment = (
await db_session.execute(
select(CompanyEnrichment).where(CompanyEnrichment.company_id == new_company.id)
)
).scalar_one()
assert new_enrichment.data == {"products": [{"name": "Widget"}]}
new_key = (
await db_session.execute(select(UserApiKey).where(UserApiKey.user_id == new_user.id))
).scalar_one()
assert new_key.provider == ApiKeyProvider.ANTHROPIC
assert new_key.encrypted_key == "encrypted-blob" # copied as-is, not re-encrypted
# The original owner's data must be completely untouched.
original_source = (
await db_session.execute(select(Source).where(Source.company_id == company.id))
).scalar_one()
assert original_source.id != new_source.id
destination = (await NotificationDestinationRepository(db_session).list_for_user(new_user.id))[
0
]
assert destination.destination_value == email
assert destination.company_links[0].company_id == new_company.id
@pytest.mark.asyncio
async def test_apply_if_pending_without_make_admin_leaves_user_a_regular_account(
db_session, settings
):
owner = await _make_user(db_session)
await _build_full_company(db_session, owner)
email = f"invitee-{uuid.uuid4().hex[:10]}@example.com"
await provisioning_service.create_pending(db_session, email, owner.id, make_admin=False)
new_user = await _make_user(db_session, email=email)
await provisioning_service.apply_if_pending(db_session, new_user)
assert new_user.is_admin is False
@pytest.mark.asyncio
async def test_delete_pending_removes_an_unconsumed_entry(db_session, settings):
owner = await _make_user(db_session)
email = f"pending-{uuid.uuid4().hex[:10]}@example.com"
record = await provisioning_service.create_pending(db_session, email, owner.id, False)
await provisioning_service.delete_pending(db_session, record.id)
assert await PendingProvisioningRepository(db_session).get_by_email(email) is None
@pytest.mark.asyncio
async def test_delete_pending_404s_for_an_unknown_id(db_session, settings):
with pytest.raises(NotFoundError):
await provisioning_service.delete_pending(db_session, uuid.uuid4())
def test_verify_email_applies_a_matching_pending_entry_end_to_end(client, settings):
"""The verify-email endpoint, not registration, is what actually
triggers provisioning - see auth_service.verify_email's docstring."""
owner_email = f"owner-{uuid.uuid4().hex[:10]}@example.com"
client.post(
"/api/v1/auth/register",
json={"email": owner_email, "password": "correct-horse-1", "display_name": "Owner"},
)
invitee_email = f"invitee-{uuid.uuid4().hex[:10]}@example.com"
async def _setup() -> None:
async with get_sessionmaker()() as db:
owner = await UserRepository(db).get_by_email(owner_email)
await _build_full_company(db, owner)
await provisioning_service.create_pending(db, invitee_email, owner.id, make_admin=True)
asyncio.run(_setup())
register_resp = client.post(
"/api/v1/auth/register",
json={"email": invitee_email, "password": "correct-horse-1", "display_name": "Invitee"},
)
assert register_resp.status_code == 201
assert register_resp.json()["is_admin"] is False # not yet - only verifying triggers it
code = "424242"
async def _issue_code() -> None:
async with get_sessionmaker()() as db:
db.add(
EmailCode(
user_id=uuid.UUID(register_resp.json()["id"]),
purpose=EmailCodePurpose.VERIFY_EMAIL,
code_hash=hash_email_code(code),
expires_at=datetime.now(UTC) + timedelta(hours=1),
)
)
await db.commit()
asyncio.run(_issue_code())
verify_resp = client.post(
"/api/v1/auth/verify-email", json={"email": invitee_email, "code": code}
)
assert verify_resp.status_code == 204
login = client.post(
"/api/v1/auth/login", json={"email": invitee_email, "password": "correct-horse-1"}
).json()
headers = {"Authorization": f"Bearer {login['access_token']}"}
me = client.get("/api/v1/auth/me", headers=headers).json()
assert me["is_admin"] is True
companies = client.get("/api/v1/companies", headers=headers).json()
assert len(companies) == 1
assert companies[0]["name"] == "Acme Corp"
def test_pending_provisioning_admin_endpoints_require_admin(client):
headers = _register_and_login_plain(client)
resp = client.get("/api/v1/system/pending-provisioning", headers=headers)
assert resp.status_code == 403
def _register_and_login_plain(client) -> dict[str, str]:
email = f"plain-{uuid.uuid4().hex[:10]}@example.com"
client.post(
"/api/v1/auth/register",
json={"email": email, "password": "correct-horse-1", "display_name": "Plain"},
)
tokens = client.post(
"/api/v1/auth/login", json={"email": email, "password": "correct-horse-1"}
).json()
return {"Authorization": f"Bearer {tokens['access_token']}"}
+283
View File
@@ -0,0 +1,283 @@
"""_backfill_from_enrichment / _mirror_swot_into_flat_lists: company_enrichment
data (products/recent_updates/funding) and the model's own SWOT output are
direct, unambiguous sources for products_and_services/recent_developments/
financial_signals/risks/opportunities - no additional LLM judgment required.
Confirmed live that a real Anthropic call can still leave these fields empty
despite explicit prompt instructions to populate them, so this backfill
guarantees the data surfaces regardless of what the model chose to do."""
from __future__ import annotations
from app.prompts.report_generation import ReportContent, SwotAnalysis
from app.prompts.schemas import Finding
from app.services.report_service import (
_backfill_from_enrichment,
_backfill_reflective_sections,
_mirror_swot_into_flat_lists,
)
def _empty_report() -> ReportContent:
return ReportContent(
executive_summary="",
company_overview="",
market_positioning="",
customer_sentiment="",
competitor_comparison="",
swot=SwotAnalysis(),
methodology="",
limitations="",
)
def test_backfills_products_from_enrichment_when_model_left_it_empty():
report = _empty_report()
enrichment = {
"products": [
{"name": "Acme Pay", "category": "Payments", "description": "Accept cards online."},
{"name": "Acme Billing", "category": "Billing", "description": "Recurring invoices."},
]
}
_backfill_from_enrichment(report, enrichment)
assert [f.headline for f in report.products_and_services] == ["Acme Pay", "Acme Billing"]
assert report.products_and_services[0].summary == "Accept cards online."
assert report.products_and_services[0].confidence == "confirmed"
assert report.products_and_services[0].evidence == []
def test_backfills_recent_developments_from_enrichment_updates():
report = _empty_report()
enrichment = {
"recent_updates": [
{
"url": "https://acme.example/blog/launch",
"date": "2026-01-01",
"text": "Acme launches new dashboard",
"type": "blog",
}
]
}
_backfill_from_enrichment(report, enrichment)
assert len(report.recent_developments) == 1
finding = report.recent_developments[0]
assert finding.headline == "Acme launches new dashboard"
assert finding.date == "2026-01-01"
assert finding.evidence[0].url == "https://acme.example/blog/launch"
def test_does_not_overwrite_findings_the_model_already_produced():
report = _empty_report()
report.products_and_services = [
Finding(headline="Model-provided product", summary="From the model itself")
]
enrichment = {"products": [{"name": "Should not appear", "description": "..."}]}
_backfill_from_enrichment(report, enrichment)
assert len(report.products_and_services) == 1
assert report.products_and_services[0].headline == "Model-provided product"
def test_no_enrichment_data_leaves_lists_empty():
report = _empty_report()
_backfill_from_enrichment(report, None)
assert report.products_and_services == []
assert report.recent_developments == []
def test_skips_entries_missing_the_required_key():
report = _empty_report()
enrichment = {
"products": [{"category": "No name here"}],
"recent_updates": [{"url": "https://acme.example", "date": "2026-01-01"}],
}
_backfill_from_enrichment(report, enrichment)
assert report.products_and_services == []
assert report.recent_developments == []
def test_backfills_financial_signals_from_funding_rounds_and_total():
report = _empty_report()
enrichment = {
"funding": {
"total_raised": "9810000000",
"rounds": [
{
"date": "2026-02-01",
"amount": None,
"investors": ["Thrive Capital", "Coatue Management"],
"round_name": "SECONDARY_SALE",
},
{
"date": "2023-03-01",
"amount": "6870000000",
"investors": ["Thrive Capital", "Andreessen Horowitz"],
"round_name": "SERIES_I",
},
],
}
}
_backfill_from_enrichment(report, enrichment)
assert len(report.financial_signals) == 3
assert report.financial_signals[0].headline == "Total funding raised: $9,810,000,000"
assert report.financial_signals[1].headline == "Secondary Sale"
assert "Thrive Capital" in report.financial_signals[1].summary
assert report.financial_signals[2].headline == "Series I - $6,870,000,000"
assert report.financial_signals[2].date == "2023-03-01"
def test_no_funding_data_leaves_financial_signals_empty():
report = _empty_report()
_backfill_from_enrichment(report, {"products": []})
assert report.financial_signals == []
def test_does_not_overwrite_financial_signals_the_model_already_produced():
report = _empty_report()
report.financial_signals = [Finding(headline="Model-provided signal", summary="From the model")]
enrichment = {"funding": {"total_raised": "1000", "rounds": []}}
_backfill_from_enrichment(report, enrichment)
assert len(report.financial_signals) == 1
assert report.financial_signals[0].headline == "Model-provided signal"
def test_mirrors_swot_threats_into_risks_when_risks_empty():
report = _empty_report()
report.swot = SwotAnalysis(threats=["Regulatory scrutiny", "New entrants"])
_mirror_swot_into_flat_lists(report)
assert report.risks == ["Regulatory scrutiny", "New entrants"]
def test_mirrors_swot_opportunities_into_opportunities_when_empty():
report = _empty_report()
report.swot = SwotAnalysis(opportunities=["Expand into new markets"])
_mirror_swot_into_flat_lists(report)
assert report.opportunities == ["Expand into new markets"]
def test_does_not_overwrite_risks_or_opportunities_the_model_already_produced():
report = _empty_report()
report.risks = ["Model-provided risk"]
report.opportunities = ["Model-provided opportunity"]
report.swot = SwotAnalysis(threats=["Should not appear"], opportunities=["Should not appear"])
_mirror_swot_into_flat_lists(report)
assert report.risks == ["Model-provided risk"]
assert report.opportunities == ["Model-provided opportunity"]
def test_empty_swot_leaves_risks_and_opportunities_empty():
report = _empty_report()
_mirror_swot_into_flat_lists(report)
assert report.risks == []
assert report.opportunities == []
def test_backfills_strategic_initiatives_from_specialties():
report = _empty_report()
enrichment = {"specialties": ["Payment Processing", "Billing Models"]}
_backfill_reflective_sections(
report, enrichment=enrichment, documents=[], changes=[], company_name="Acme"
)
assert [f.headline for f in report.strategic_initiatives] == [
"Focus area: Payment Processing",
"Focus area: Billing Models",
]
assert report.strategic_initiatives[0].confidence == "possible"
def test_regulatory_signals_get_an_insufficient_evidence_placeholder_when_empty():
report = _empty_report()
_backfill_reflective_sections(
report, enrichment=None, documents=[], changes=[], company_name="Acme"
)
assert len(report.regulatory_and_legal_signals) == 1
assert report.regulatory_and_legal_signals[0].confidence == "insufficient_evidence"
def test_unknowns_reflect_actual_gaps_in_the_evidence_set():
report = _empty_report()
enrichment = {"funding": {}, "leadership_team": [], "customers": []}
_backfill_reflective_sections(
report, enrichment=enrichment, documents=[], changes=[], company_name="Acme"
)
assert any("financial" in u.lower() for u in report.unknowns_and_missing_data)
assert any("leadership" in u.lower() for u in report.unknowns_and_missing_data)
assert any("customers" in u.lower() for u in report.unknowns_and_missing_data)
assert any("source documents" in u.lower() for u in report.unknowns_and_missing_data)
assert any("changes" in u.lower() for u in report.unknowns_and_missing_data)
def test_unknowns_omit_gaps_that_are_actually_covered():
report = _empty_report()
enrichment = {
"funding": {"total_raised": "1000"},
"leadership_team": [{"name": "Jane"}],
"customers": [{"name": "Acme Corp"}],
}
_backfill_reflective_sections(
report,
enrichment=enrichment,
documents=[{"id": "d1"}],
changes=[{"id": "c1"}],
company_name="Acme",
)
assert report.unknowns_and_missing_data == []
def test_monitoring_recommendations_populated_when_model_left_empty():
report = _empty_report()
_backfill_reflective_sections(
report, enrichment=None, documents=[], changes=[], company_name="Acme"
)
assert len(report.monitoring_recommendations) >= 2
assert any("Acme" in r for r in report.monitoring_recommendations)
def test_reflective_sections_do_not_overwrite_model_output():
report = _empty_report()
report.strategic_initiatives = [Finding(headline="Model theme", summary="From the model")]
report.regulatory_and_legal_signals = [Finding(headline="Model signal", summary="From model")]
report.unknowns_and_missing_data = ["Model-noted gap"]
report.monitoring_recommendations = ["Model recommendation"]
enrichment = {"specialties": ["Should not appear"]}
_backfill_reflective_sections(
report, enrichment=enrichment, documents=[], changes=[], company_name="Acme"
)
assert report.strategic_initiatives[0].headline == "Model theme"
assert report.regulatory_and_legal_signals[0].headline == "Model signal"
assert report.unknowns_and_missing_data == ["Model-noted gap"]
assert report.monitoring_recommendations == ["Model recommendation"]
+101 -30
View File
@@ -11,6 +11,7 @@ import {
ChevronRight, ChevronRight,
Loader2, Loader2,
Plus, Plus,
RefreshCw,
} from "lucide-react"; } from "lucide-react";
import { CompanyPillLink } from "@/components/ui/company-pill"; import { CompanyPillLink } from "@/components/ui/company-pill";
import { CopyableEmail } from "@/components/ui/copyable-email"; import { CopyableEmail } from "@/components/ui/copyable-email";
@@ -23,6 +24,7 @@ import {
useCompanies, useCompanies,
useCompany, useCompany,
useDeleteCompany, useDeleteCompany,
useRefreshEnrichmentSection,
useSetCompanyPaused, useSetCompanyPaused,
useUpdateMonitorConfiguration, useUpdateMonitorConfiguration,
} from "@/hooks/use-companies"; } from "@/hooks/use-companies";
@@ -37,7 +39,7 @@ import {
import { useCompanyRuns, useRunCompanyNow } from "@/hooks/use-monitoring-runs"; import { useCompanyRuns, useRunCompanyNow } from "@/hooks/use-monitoring-runs";
import { useCompanyReports, useGenerateReport } from "@/hooks/use-reports"; import { useCompanyReports, useGenerateReport } from "@/hooks/use-reports";
import { useSnapshots } from "@/hooks/use-snapshots"; import { useSnapshots } from "@/hooks/use-snapshots";
import type { MonitoringFrequency } from "@/lib/types"; import type { EnrichmentSection, MonitoringFrequency } from "@/lib/types";
import { import {
FREQUENCY_LABELS, FREQUENCY_LABELS,
MONITORING_FREQUENCIES, MONITORING_FREQUENCIES,
@@ -149,6 +151,30 @@ function EnrichmentErrorSummary({
); );
} }
function RefreshSectionButton({
section,
label,
mutation,
}: {
section: EnrichmentSection;
label: string;
mutation: ReturnType<typeof useRefreshEnrichmentSection>;
}) {
const isRefreshing = mutation.isPending && mutation.variables === section;
return (
<button
type="button"
onClick={() => mutation.mutate(section)}
disabled={isRefreshing}
title={`Re-fetch ${label} from NinjaPear`}
className="focus-ring mt-2 inline-flex items-center gap-1.5 rounded-md border border-slate-300 px-2 py-1 text-xs font-medium text-slate-700 transition-colors duration-150 hover:bg-slate-50 disabled:opacity-60"
>
<RefreshCw className={`h-3 w-3 ${isRefreshing ? "animate-spin" : ""}`} aria-hidden />
{isRefreshing ? "Refreshing…" : "Refresh"}
</button>
);
}
export default function CompanyDetailPage({ params }: { params: Promise<{ id: string }> }) { export default function CompanyDetailPage({ params }: { params: Promise<{ id: string }> }) {
const { id } = use(params); const { id } = use(params);
return ( return (
@@ -176,6 +202,7 @@ function CompanyDetailPageInner({ id }: { id: string }) {
const deleteSource = useDeleteSource(id); const deleteSource = useDeleteSource(id);
const testSource = useTestSource(id); const testSource = useTestSource(id);
const updateSource = useUpdateSource(id); const updateSource = useUpdateSource(id);
const refreshEnrichmentSection = useRefreshEnrichmentSection(id);
const { data: runs, isLoading: runsLoading } = useCompanyRuns(id); const { data: runs, isLoading: runsLoading } = useCompanyRuns(id);
const runNow = useRunCompanyNow(id); const runNow = useRunCompanyNow(id);
const { data: reports, isLoading: reportsLoading } = useCompanyReports(id); const { data: reports, isLoading: reportsLoading } = useCompanyReports(id);
@@ -532,7 +559,16 @@ function CompanyDetailPageInner({ id }: { id: string }) {
| { total_raised?: string; rounds?: Record<string, unknown>[] } | { total_raised?: string; rounds?: Record<string, unknown>[] }
| undefined; | undefined;
if (!funding || (!funding.total_raised && !funding.rounds?.length)) { if (!funding || (!funding.total_raised && !funding.rounds?.length)) {
return <p className="mt-2 text-slate-500">No funding data found.</p>; return (
<div className="animate-fade-in">
<p className="mt-2 text-slate-500">No funding data found.</p>
<RefreshSectionButton
section="funding"
label="funding data"
mutation={refreshEnrichmentSection}
/>
</div>
);
} }
return ( return (
<> <>
@@ -574,7 +610,14 @@ function CompanyDetailPageInner({ id }: { id: string }) {
<h2 className="font-medium text-slate-700">Leadership team</h2> <h2 className="font-medium text-slate-700">Leadership team</h2>
{!Array.isArray(company.enrichment.data.leadership_team) || {!Array.isArray(company.enrichment.data.leadership_team) ||
(company.enrichment.data.leadership_team as unknown[]).length === 0 ? ( (company.enrichment.data.leadership_team as unknown[]).length === 0 ? (
<div className="animate-fade-in">
<p className="mt-2 text-slate-500">No leadership team data found.</p> <p className="mt-2 text-slate-500">No leadership team data found.</p>
<RefreshSectionButton
section="details"
label="leadership team"
mutation={refreshEnrichmentSection}
/>
</div>
) : ( ) : (
<ul className="mt-2"> <ul className="mt-2">
{( {(
@@ -613,37 +656,18 @@ function CompanyDetailPageInner({ id }: { id: string }) {
</section> </section>
<div className="grid gap-4 sm:grid-cols-2"> <div className="grid gap-4 sm:grid-cols-2">
<section className="rounded-lg border border-slate-200 bg-white p-5 text-sm">
<h2 className="font-medium text-slate-700">Customers</h2>
{!Array.isArray(company.enrichment.data.customers) ||
(company.enrichment.data.customers as unknown[]).length === 0 ? (
<p className="mt-2 text-slate-500">None found.</p>
) : (
<ul className="mt-2 flex flex-wrap gap-2">
{(company.enrichment.data.customers as Record<string, unknown>[]).map(
(c, i) => {
const name = String(c.name ?? "");
if (!name) return null;
return (
<li key={i}>
<CompanyPillLink
name={name}
monitoredByName={monitoredByName}
returnTo={returnTo}
/>
</li>
);
},
)}
</ul>
)}
</section>
<section className="rounded-lg border border-slate-200 bg-white p-5 text-sm"> <section className="rounded-lg border border-slate-200 bg-white p-5 text-sm">
<h2 className="font-medium text-slate-700">Products</h2> <h2 className="font-medium text-slate-700">Products</h2>
{!Array.isArray(company.enrichment.data.products) || {!Array.isArray(company.enrichment.data.products) ||
(company.enrichment.data.products as unknown[]).length === 0 ? ( (company.enrichment.data.products as unknown[]).length === 0 ? (
<div className="animate-fade-in">
<p className="mt-2 text-slate-500">None found.</p> <p className="mt-2 text-slate-500">None found.</p>
<RefreshSectionButton
section="products"
label="products"
mutation={refreshEnrichmentSection}
/>
</div>
) : ( ) : (
<ul className="mt-2 space-y-1.5"> <ul className="mt-2 space-y-1.5">
{(company.enrichment.data.products as Record<string, unknown>[]).map( {(company.enrichment.data.products as Record<string, unknown>[]).map(
@@ -662,14 +686,19 @@ function CompanyDetailPageInner({ id }: { id: string }) {
</ul> </ul>
)} )}
</section> </section>
</div>
<div className="grid gap-4 sm:grid-cols-2">
<section className="rounded-lg border border-slate-200 bg-white p-5 text-sm"> <section className="rounded-lg border border-slate-200 bg-white p-5 text-sm">
<h2 className="font-medium text-slate-700">Recent updates</h2> <h2 className="font-medium text-slate-700">Recent updates</h2>
{!Array.isArray(company.enrichment.data.recent_updates) || {!Array.isArray(company.enrichment.data.recent_updates) ||
(company.enrichment.data.recent_updates as unknown[]).length === 0 ? ( (company.enrichment.data.recent_updates as unknown[]).length === 0 ? (
<div className="animate-fade-in">
<p className="mt-2 text-slate-500">None found.</p> <p className="mt-2 text-slate-500">None found.</p>
<RefreshSectionButton
section="updates"
label="recent updates"
mutation={refreshEnrichmentSection}
/>
</div>
) : ( ) : (
<ul className="mt-2"> <ul className="mt-2">
{( {(
@@ -693,6 +722,41 @@ function CompanyDetailPageInner({ id }: { id: string }) {
</ul> </ul>
)} )}
</section> </section>
</div>
<div className="grid gap-4 sm:grid-cols-2">
<section className="rounded-lg border border-slate-200 bg-white p-5 text-sm">
<h2 className="font-medium text-slate-700">Customers</h2>
{!Array.isArray(company.enrichment.data.customers) ||
(company.enrichment.data.customers as unknown[]).length === 0 ? (
<div className="animate-fade-in">
<p className="mt-2 text-slate-500">None found.</p>
<RefreshSectionButton
section="customers"
label="customers"
mutation={refreshEnrichmentSection}
/>
</div>
) : (
<ul className="mt-2 flex flex-wrap gap-2">
{(company.enrichment.data.customers as Record<string, unknown>[]).map(
(c, i) => {
const name = String(c.name ?? "");
if (!name) return null;
return (
<li key={i}>
<CompanyPillLink
name={name}
monitoredByName={monitoredByName}
returnTo={returnTo}
/>
</li>
);
},
)}
</ul>
)}
</section>
<section className="rounded-lg border border-slate-200 bg-white p-5 text-sm"> <section className="rounded-lg border border-slate-200 bg-white p-5 text-sm">
<h2 className="font-medium text-slate-700"> <h2 className="font-medium text-slate-700">
@@ -701,7 +765,14 @@ function CompanyDetailPageInner({ id }: { id: string }) {
</h2> </h2>
{!Array.isArray(company.enrichment.data.competitors) || {!Array.isArray(company.enrichment.data.competitors) ||
(company.enrichment.data.competitors as unknown[]).length === 0 ? ( (company.enrichment.data.competitors as unknown[]).length === 0 ? (
<div className="animate-fade-in">
<p className="mt-2 text-slate-500">None found.</p> <p className="mt-2 text-slate-500">None found.</p>
<RefreshSectionButton
section="competitors"
label="competitors"
mutation={refreshEnrichmentSection}
/>
</div>
) : ( ) : (
<ul className="mt-2 flex flex-wrap gap-2"> <ul className="mt-2 flex flex-wrap gap-2">
{( {(
+18
View File
@@ -15,6 +15,7 @@ export default function DashboardLayout({ children }: { children: React.ReactNod
const logout = useLogout(); const logout = useLogout();
const requiresLogin = systemStatus ? !isLocalConvenience(systemStatus) : false; const requiresLogin = systemStatus ? !isLocalConvenience(systemStatus) : false;
const mustChangePassword = user?.must_change_password ?? false;
useEffect(() => { useEffect(() => {
if (requiresLogin && !isLoading && isError) { if (requiresLogin && !isLoading && isError) {
@@ -22,6 +23,12 @@ export default function DashboardLayout({ children }: { children: React.ReactNod
} }
}, [requiresLogin, isLoading, isError, router]); }, [requiresLogin, isLoading, isError, router]);
useEffect(() => {
if (mustChangePassword) {
router.replace("/change-password");
}
}, [mustChangePassword, router]);
if (requiresLogin && (isLoading || isError)) { if (requiresLogin && (isLoading || isError)) {
return ( return (
<div className="flex min-h-screen items-center justify-center text-sm text-slate-500"> <div className="flex min-h-screen items-center justify-center text-sm text-slate-500">
@@ -30,6 +37,17 @@ export default function DashboardLayout({ children }: { children: React.ReactNod
); );
} }
// Every API call except /auth/me, /auth/change-password, and /auth/logout
// 403s server-side while this is set (app/auth/dependencies.py) - this is
// just the matching frontend redirect, not the actual enforcement.
if (mustChangePassword) {
return (
<div className="flex min-h-screen items-center justify-center text-sm text-slate-500">
Redirecting
</div>
);
}
return ( return (
<div className="min-h-screen bg-slate-50"> <div className="min-h-screen bg-slate-50">
<LocalModeBanner /> <LocalModeBanner />
+120
View File
@@ -5,6 +5,7 @@ import { useState } from "react";
import { import {
Bell, Bell,
CheckCircle2, CheckCircle2,
Database,
KeyRound, KeyRound,
Loader2, Loader2,
Mail, Mail,
@@ -24,8 +25,10 @@ import {
import { authErrorMessage } from "@/hooks/use-auth"; import { authErrorMessage } from "@/hooks/use-auth";
import { import {
useAcceptUnbanRequest, useAcceptUnbanRequest,
useCreateDbViewerSession,
useCreateIpBan, useCreateIpBan,
useCurrentUser, useCurrentUser,
useDeleteAccount,
useDeleteIpBan, useDeleteIpBan,
useIpBans, useIpBans,
useRejectUnbanRequest, useRejectUnbanRequest,
@@ -48,6 +51,7 @@ import { FormField } from "@/components/ui/form-field";
import { Select } from "@/components/ui/select"; import { Select } from "@/components/ui/select";
import { SystemSecretRow } from "@/components/ui/system-secret-row"; import { SystemSecretRow } from "@/components/ui/system-secret-row";
import { UserApiKeyRow } from "@/components/ui/user-api-key-row"; import { UserApiKeyRow } from "@/components/ui/user-api-key-row";
import { isLocalConvenience } from "@/lib/auth";
import { formatDateTime } from "@/lib/format"; import { formatDateTime } from "@/lib/format";
import { import {
SEVERITY_LABELS, SEVERITY_LABELS,
@@ -663,6 +667,119 @@ function IpBansBox() {
); );
} }
function DatabaseViewerBox() {
const { data: systemStatus } = useSystemStatus();
const createSession = useCreateDbViewerSession();
const [error, setError] = useState<string | null>(null);
const isLocal = systemStatus ? isLocalConvenience(systemStatus) : false;
const dbViewerUrl = process.env.NEXT_PUBLIC_DB_VIEWER_URL;
const handleOpen = async () => {
setError(null);
try {
const { token } = await createSession.mutateAsync();
window.open(`${dbViewerUrl}/_auth?token=${encodeURIComponent(token)}`, "_blank", "noopener,noreferrer");
} catch {
setError("Couldn't open the database viewer. Try again.");
}
};
return (
<div className="rounded-lg border border-slate-200 bg-white p-6">
<div className="flex items-center gap-2 text-sm font-semibold text-slate-700">
<Database className="h-4 w-4" aria-hidden /> Database
</div>
<p className="mt-1 text-sm text-slate-500">
Open a web-based Postgres client (Adminer) in a new tab to view and edit rows directly.
PostgreSQL is pre-selected - just enter the database password to sign in. Local developers
and server admins only.
</p>
<div className="mt-4">
{isLocal ? (
<a
href="http://localhost:8081/?pgsql=postgres&username=ciagent"
target="_blank"
rel="noopener noreferrer"
className="focus-ring inline-flex items-center gap-1.5 rounded-md bg-slate-800 px-3 py-1.5 text-xs font-semibold text-white transition-colors duration-200 hover:bg-slate-900"
>
<Database className="h-3.5 w-3.5" aria-hidden /> Open database viewer
</a>
) : dbViewerUrl ? (
<ActionButton
onClick={handleOpen}
pending={createSession.isPending}
colorClass="bg-slate-800 hover:bg-slate-900"
icon={Database}
label="Open database viewer"
/>
) : (
<p className="text-sm text-slate-500">Not configured for this deployment.</p>
)}
{error && <p className="mt-2 text-xs text-red-600">{error}</p>}
</div>
</div>
);
}
function DeleteAccountBox() {
const { data: systemStatus } = useSystemStatus();
const [password, setPassword] = useState("");
const deleteAccount = useDeleteAccount();
const isLocal = systemStatus ? isLocalConvenience(systemStatus) : false;
// The local-dev bypass account has no password_hash at all (it never
// registers/logs in) - there's nothing to type to confirm deletion, and
// deleting it would just get silently re-created on the next request.
if (isLocal) return null;
return (
<div className="rounded-lg border border-red-200 bg-red-50 p-6">
<div className="flex items-center gap-2 text-sm font-semibold text-red-700">
<Trash2 className="h-4 w-4" aria-hidden /> Delete account
</div>
<p className="mt-1 text-sm text-red-700/80">
Permanently deletes your account and everything in it - companies, monitoring history,
reports, and API keys. This can&apos;t be undone.
</p>
<form
onSubmit={(e) => {
e.preventDefault();
deleteAccount.mutate({ password });
}}
className="mt-4 flex flex-wrap items-end gap-2"
>
<div className="min-w-[220px] flex-1">
<label htmlFor="delete-account-password" className="block text-sm font-medium text-red-700">
Confirm your password
</label>
<input
id="delete-account-password"
type="password"
autoComplete="current-password"
value={password}
onChange={(e) => setPassword(e.target.value)}
className="focus-ring mt-1 block w-full rounded-md border border-red-300 px-3 py-2 text-sm text-slate-900 shadow-sm transition-colors duration-150"
/>
</div>
<button
type="submit"
disabled={!password || deleteAccount.isPending}
className="focus-ring inline-flex shrink-0 items-center gap-1.5 rounded-md bg-red-600 px-3 py-2 text-sm font-semibold text-white transition-colors duration-200 hover:bg-red-700 disabled:cursor-not-allowed disabled:opacity-50"
>
{deleteAccount.isPending && <Loader2 className="h-4 w-4 animate-spin" aria-hidden />}
{deleteAccount.isPending ? "Deleting…" : "Delete account"}
</button>
</form>
{deleteAccount.isError && (
<p className="mt-2 text-sm text-red-700" role="alert">
{authErrorMessage(deleteAccount.error)}
</p>
)}
</div>
);
}
export default function SettingsPage() { export default function SettingsPage() {
const { data: user } = useCurrentUser(); const { data: user } = useCurrentUser();
const { data: systemStatus } = useSystemStatus(); const { data: systemStatus } = useSystemStatus();
@@ -759,9 +876,12 @@ export default function SettingsPage() {
<> <>
<ServerSecretsBox /> <ServerSecretsBox />
<IpBansBox /> <IpBansBox />
<DatabaseViewerBox />
<LoggingBox /> <LoggingBox />
</> </>
)} )}
<DeleteAccountBox />
</div> </div>
); );
} }
+100
View File
@@ -0,0 +1,100 @@
"use client";
import { zodResolver } from "@hookform/resolvers/zod";
import { useRouter } from "next/navigation";
import { useForm } from "react-hook-form";
import { KeyRound, Loader2 } from "lucide-react";
import { z } from "zod";
import { PasswordField } from "@/components/ui/password-field";
import { PasswordStrengthMeter } from "@/components/ui/password-strength-meter";
import { authErrorMessage, useChangePassword } from "@/hooks/use-auth";
const changePasswordSchema = z.object({
currentPassword: z.string().min(1, "Enter your current password"),
newPassword: z
.string()
.min(10, "Must be at least 10 characters")
.refine((v) => /[a-zA-Z]/.test(v) && /\d/.test(v), {
message: "Must contain at least one letter and one digit",
}),
});
type ChangePasswordForm = z.infer<typeof changePasswordSchema>;
export default function ChangePasswordPage() {
const router = useRouter();
const changePassword = useChangePassword();
const {
register,
handleSubmit,
watch,
formState: { errors },
} = useForm<ChangePasswordForm>({ resolver: zodResolver(changePasswordSchema) });
const onSubmit = handleSubmit(async (values) => {
try {
await changePassword.mutateAsync({
current_password: values.currentPassword,
new_password: values.newPassword,
});
router.replace("/dashboard");
} catch {
// Surfaced via changePassword.isError below.
}
});
return (
<main className="flex min-h-screen items-center justify-center bg-slate-50 px-4 py-12">
<div className="w-full max-w-sm animate-fade-in overflow-hidden rounded-lg border border-slate-200 bg-white shadow-sm">
<div className="h-1.5 bg-gradient-to-r from-brand-500 via-brand-600 to-brand-700" />
<div className="p-8">
<div className="flex items-center gap-3">
<span className="flex h-9 w-9 shrink-0 items-center justify-center rounded-full bg-brand-50 text-brand-600">
<KeyRound className="h-4.5 w-4.5" aria-hidden />
</span>
<div>
<h1 className="text-xl font-semibold text-slate-900">Change your password</h1>
<p className="text-sm text-slate-600">
An admin requires a new password before you can continue.
</p>
</div>
</div>
<form onSubmit={onSubmit} className="mt-6 space-y-4" noValidate>
<PasswordField
label="Current password"
autoComplete="current-password"
{...register("currentPassword")}
error={errors.currentPassword?.message}
/>
<div>
<PasswordField
label="New password"
autoComplete="new-password"
hint="At least 10 characters, with a letter and a digit."
{...register("newPassword")}
error={errors.newPassword?.message}
/>
<PasswordStrengthMeter password={watch("newPassword") ?? ""} />
</div>
{changePassword.isError && (
<p className="animate-fade-in text-sm text-red-600" role="alert">
{authErrorMessage(changePassword.error)}
</p>
)}
<button
type="submit"
disabled={changePassword.isPending}
className="focus-ring inline-flex w-full items-center justify-center gap-2 rounded-md bg-brand-600 px-4 py-2 text-sm font-semibold text-white transition-colors duration-200 hover:bg-brand-700 disabled:cursor-not-allowed disabled:opacity-60"
>
{changePassword.isPending && <Loader2 className="h-4 w-4 animate-spin" aria-hidden />}
{changePassword.isPending ? "Updating…" : "Update password"}
</button>
</form>
</div>
</div>
</main>
);
}
+35
View File
@@ -5,7 +5,9 @@ import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
import { ApiError, api, clearTokens, getRefreshToken, setTokens } from "@/lib/api-client"; import { ApiError, api, clearTokens, getRefreshToken, setTokens } from "@/lib/api-client";
import type { import type {
BanIpPayload, BanIpPayload,
ChangePasswordPayload,
ConfirmPasswordResetPayload, ConfirmPasswordResetPayload,
DeleteAccountPayload,
LoginPayload, LoginPayload,
RegisterPayload, RegisterPayload,
RequestPasswordResetPayload, RequestPasswordResetPayload,
@@ -47,6 +49,12 @@ export function useSetSystemSecret() {
}); });
} }
export function useCreateDbViewerSession() {
return useMutation({
mutationFn: () => api.createDbViewerSession(),
});
}
export function useSystemLogs() { export function useSystemLogs() {
return useQuery({ return useQuery({
queryKey: ["system-logs"], queryKey: ["system-logs"],
@@ -104,6 +112,33 @@ export function useLogout() {
}); });
} }
export function useDeleteAccount() {
const router = useRouter();
const queryClient = useQueryClient();
return useMutation({
mutationFn: (payload: DeleteAccountPayload) => api.deleteAccount(payload),
onSuccess: () => {
// Same reasoning as useLogout: we know for certain the account is
// gone, so clear and navigate immediately rather than waiting on a
// background refetch of ["me"] to fail.
clearTokens();
queryClient.clear();
router.replace("/");
},
});
}
export function useChangePassword() {
const queryClient = useQueryClient();
return useMutation({
mutationFn: (payload: ChangePasswordPayload) => api.changePassword(payload),
onSuccess: (tokens) => {
setTokens(tokens.access_token, tokens.refresh_token);
return queryClient.invalidateQueries({ queryKey: ["me"] });
},
});
}
export function useVerifyEmail() { export function useVerifyEmail() {
return useMutation({ return useMutation({
mutationFn: (payload: VerifyEmailPayload) => api.verifyEmail(payload), mutationFn: (payload: VerifyEmailPayload) => api.verifyEmail(payload),
+12
View File
@@ -6,6 +6,7 @@ import type {
CompanyCreatePayload, CompanyCreatePayload,
CompanyResponse, CompanyResponse,
CompanyUpdatePayload, CompanyUpdatePayload,
EnrichmentSection,
MonitorConfigurationUpdatePayload, MonitorConfigurationUpdatePayload,
} from "@/lib/types"; } from "@/lib/types";
@@ -67,6 +68,17 @@ export function useSetCompanyPaused() {
}); });
} }
export function useRefreshEnrichmentSection(companyId: string) {
const queryClient = useQueryClient();
return useMutation({
mutationFn: (section: EnrichmentSection) =>
api.refreshEnrichmentSection(companyId, section),
onSuccess: (company) => {
queryClient.setQueryData(["companies", companyId], company);
},
});
}
export function useUpdateMonitorConfiguration(companyId: string) { export function useUpdateMonitorConfiguration(companyId: string) {
const queryClient = useQueryClient(); const queryClient = useQueryClient();
return useMutation({ return useMutation({
+25
View File
@@ -10,9 +10,13 @@ import type {
CompanyResponse, CompanyResponse,
CompanyUpdatePayload, CompanyUpdatePayload,
ConfirmPasswordResetPayload, ConfirmPasswordResetPayload,
ChangePasswordPayload,
DashboardAnalytics, DashboardAnalytics,
DbViewerSessionResponse,
DeleteAccountPayload,
DiscoverCompanyRequest, DiscoverCompanyRequest,
DiscoveredCompanyProfile, DiscoveredCompanyProfile,
EnrichmentSection,
IpBan, IpBan,
LogEntry, LogEntry,
LoginPayload, LoginPayload,
@@ -166,6 +170,9 @@ export const api = {
listSystemSecrets: () => request<SystemSecretStatus[]>("/api/v1/system/secrets"), listSystemSecrets: () => request<SystemSecretStatus[]>("/api/v1/system/secrets"),
createDbViewerSession: () =>
request<DbViewerSessionResponse>("/api/v1/db-viewer/session", { method: "POST" }),
setSystemSecret: (key: string, payload: SetSystemSecretPayload) => setSystemSecret: (key: string, payload: SetSystemSecretPayload) =>
request<SystemSecretStatus>(`/api/v1/system/secrets/${key}`, { request<SystemSecretStatus>(`/api/v1/system/secrets/${key}`, {
method: "PUT", method: "PUT",
@@ -201,6 +208,18 @@ export const api = {
me: () => request<MeResponse>("/api/v1/auth/me"), me: () => request<MeResponse>("/api/v1/auth/me"),
deleteAccount: (payload: DeleteAccountPayload) =>
request<void>("/api/v1/auth/me", {
method: "DELETE",
body: JSON.stringify(payload),
}),
changePassword: (payload: ChangePasswordPayload) =>
request<TokenResponse>("/api/v1/auth/change-password", {
method: "POST",
body: JSON.stringify(payload),
}),
verifyEmail: (payload: VerifyEmailPayload) => verifyEmail: (payload: VerifyEmailPayload) =>
request<void>("/api/v1/auth/verify-email", { request<void>("/api/v1/auth/verify-email", {
method: "POST", method: "POST",
@@ -298,6 +317,12 @@ export const api = {
resumeCompany: (companyId: string) => resumeCompany: (companyId: string) =>
request<CompanyResponse>(`/api/v1/companies/${companyId}/resume`, { method: "POST" }), request<CompanyResponse>(`/api/v1/companies/${companyId}/resume`, { method: "POST" }),
refreshEnrichmentSection: (companyId: string, section: EnrichmentSection) =>
request<CompanyResponse>(
`/api/v1/companies/${companyId}/enrichment/sections/${section}/refresh`,
{ method: "POST" },
),
updateMonitorConfiguration: (companyId: string, payload: MonitorConfigurationUpdatePayload) => updateMonitorConfiguration: (companyId: string, payload: MonitorConfigurationUpdatePayload) =>
request<MonitorConfigurationResponse>(`/api/v1/companies/${companyId}/monitor`, { request<MonitorConfigurationResponse>(`/api/v1/companies/${companyId}/monitor`, {
method: "PATCH", method: "PATCH",
+22
View File
@@ -29,6 +29,19 @@ export interface SystemSecretStatus {
value: string | null; value: string | null;
} }
export interface DbViewerSessionResponse {
token: string;
}
export interface DeleteAccountPayload {
password: string;
}
export interface ChangePasswordPayload {
current_password: string;
new_password: string;
}
export interface SetSystemSecretPayload { export interface SetSystemSecretPayload {
value: string; value: string;
} }
@@ -66,6 +79,7 @@ export interface UserResponse {
timezone: string; timezone: string;
is_active: boolean; is_active: boolean;
is_admin: boolean; is_admin: boolean;
must_change_password: boolean;
} }
export interface MeResponse extends UserResponse { export interface MeResponse extends UserResponse {
@@ -207,6 +221,14 @@ export interface MonitorConfigurationUpdatePayload {
export type EnrichmentStatus = "pending" | "partial" | "complete" | "failed"; export type EnrichmentStatus = "pending" | "partial" | "complete" | "failed";
export type EnrichmentSection =
| "details"
| "funding"
| "updates"
| "competitors"
| "products"
| "customers";
export interface CompanyEnrichmentResponse { export interface CompanyEnrichmentResponse {
status: EnrichmentStatus; status: EnrichmentStatus;
data: Record<string, unknown>; data: Record<string, unknown>;
+86
View File
@@ -0,0 +1,86 @@
import { screen, waitFor } from "@testing-library/react";
import { beforeEach, describe, expect, it, vi } from "vitest";
import DashboardLayout from "@/app/(app)/layout";
import { renderWithQueryClient } from "./test-utils";
const replaceMock = vi.fn();
vi.mock("next/navigation", () => ({
useRouter: () => ({ push: vi.fn(), replace: replaceMock }),
usePathname: () => "/dashboard",
}));
function meBody(mustChangePassword: boolean) {
return {
id: "user-1",
email: "[email protected]",
display_name: "Regular User",
timezone: "America/New_York",
is_active: true,
is_admin: false,
auth_mode: "jwt",
must_change_password: mustChangePassword,
};
}
function systemStatusBody() {
return {
app_env: "development",
auth_mode: "jwt",
llm_provider: "mock",
search_provider: "mock",
sms_enabled: false,
sms_provider: "twilio",
ninjapear_configured: false,
ninjapear_credit_balance: null,
ninjapear_estimated_credits_per_company: null,
is_localhost: false,
turnstile_site_key: null,
components: [],
};
}
function mockFetchImplementation(mustChangePassword: boolean) {
return vi.fn().mockImplementation((url: string) => {
const path = url.replace("http://localhost:8000", "");
if (path === "/api/v1/auth/me") {
return Promise.resolve({ ok: true, status: 200, json: async () => meBody(mustChangePassword) });
}
if (path === "/api/v1/system/status") {
return Promise.resolve({ ok: true, status: 200, json: async () => systemStatusBody() });
}
return Promise.resolve({ ok: false, status: 404, json: async () => ({ detail: "not found" }) });
});
}
beforeEach(() => {
replaceMock.mockClear();
});
describe("DashboardLayout - must_change_password redirect", () => {
it("redirects to /change-password and withholds the page content when the flag is set", async () => {
vi.stubGlobal("fetch", mockFetchImplementation(true));
renderWithQueryClient(
<DashboardLayout>
<div>Protected dashboard content</div>
</DashboardLayout>,
);
await waitFor(() => {
expect(replaceMock).toHaveBeenCalledWith("/change-password");
});
expect(screen.queryByText(/protected dashboard content/i)).not.toBeInTheDocument();
});
it("renders normally when the flag is not set", async () => {
vi.stubGlobal("fetch", mockFetchImplementation(false));
renderWithQueryClient(
<DashboardLayout>
<div>Protected dashboard content</div>
</DashboardLayout>,
);
expect(await screen.findByText(/protected dashboard content/i)).toBeInTheDocument();
expect(replaceMock).not.toHaveBeenCalledWith("/change-password");
});
});
@@ -0,0 +1,86 @@
import { screen, waitFor } from "@testing-library/react";
import userEvent from "@testing-library/user-event";
import { beforeEach, describe, expect, it, vi } from "vitest";
import ChangePasswordPage from "@/app/change-password/page";
import { renderWithQueryClient } from "./test-utils";
const replaceMock = vi.fn();
vi.mock("next/navigation", () => ({
useRouter: () => ({ push: vi.fn(), replace: replaceMock }),
}));
beforeEach(() => {
replaceMock.mockClear();
});
describe("ChangePasswordPage", () => {
it("rejects a weak new password before submitting", async () => {
const user = userEvent.setup();
renderWithQueryClient(<ChangePasswordPage />);
await user.type(screen.getByLabelText(/current password/i), "correct-horse-1");
await user.type(screen.getByLabelText(/new password/i), "allletters");
await user.click(screen.getByRole("button", { name: /update password/i }));
await waitFor(() => {
expect(
screen.getByText(/must contain at least one letter and one digit/i),
).toBeInTheDocument();
});
});
it("submits current + new password and redirects to the dashboard on success", async () => {
const fetchMock = vi.fn().mockImplementation((url: string, init?: RequestInit) => {
const path = url.replace("http://localhost:8000", "");
if (path === "/api/v1/auth/change-password" && init?.method === "POST") {
expect(JSON.parse(init.body as string)).toEqual({
current_password: "correct-horse-1",
new_password: "brand-new-horse-2",
});
return Promise.resolve({
ok: true,
status: 200,
json: async () => ({
access_token: "new-access",
refresh_token: "new-refresh",
token_type: "bearer",
expires_in_minutes: 15,
}),
});
}
return Promise.resolve({ ok: false, status: 404, json: async () => ({ detail: "not found" }) });
});
vi.stubGlobal("fetch", fetchMock);
const user = userEvent.setup();
renderWithQueryClient(<ChangePasswordPage />);
await user.type(screen.getByLabelText(/current password/i), "correct-horse-1");
await user.type(screen.getByLabelText(/new password/i), "brand-new-horse-2");
await user.click(screen.getByRole("button", { name: /update password/i }));
await waitFor(() => {
expect(replaceMock).toHaveBeenCalledWith("/dashboard");
});
});
it("surfaces an error for the wrong current password", async () => {
vi.stubGlobal(
"fetch",
vi.fn().mockResolvedValue({
ok: false,
status: 401,
json: async () => ({ detail: "Incorrect current password" }),
}),
);
const user = userEvent.setup();
renderWithQueryClient(<ChangePasswordPage />);
await user.type(screen.getByLabelText(/current password/i), "wrong-password-1");
await user.type(screen.getByLabelText(/new password/i), "brand-new-horse-2");
await user.click(screen.getByRole("button", { name: /update password/i }));
expect(await screen.findByText(/incorrect current password/i)).toBeInTheDocument();
});
});
@@ -0,0 +1,105 @@
import { screen, waitFor } from "@testing-library/react";
import userEvent from "@testing-library/user-event";
import { beforeEach, describe, expect, it, vi } from "vitest";
import SettingsPage from "@/app/(app)/settings/page";
import { renderWithQueryClient } from "./test-utils";
vi.mock("next/navigation", () => ({
useRouter: () => ({ push: vi.fn(), replace: vi.fn() }),
}));
const ADMIN_USER = {
id: "user-1",
email: "[email protected]",
display_name: "Admin",
timezone: "America/New_York",
is_active: true,
is_admin: true,
auth_mode: "jwt",
must_change_password: false,
};
function systemStatusBody(isLocalhost: boolean, authMode: "local" | "jwt") {
return {
app_env: "development",
auth_mode: authMode,
llm_provider: "mock",
search_provider: "mock",
sms_enabled: false,
sms_provider: "twilio",
ninjapear_configured: false,
ninjapear_credit_balance: null,
ninjapear_estimated_credits_per_company: null,
is_localhost: isLocalhost,
turnstile_site_key: null,
components: [],
};
}
/** Covers every endpoint SettingsPage's admin-gated boxes touch on mount -
* empty-list/no-op responses for everything except /system/status, which
* each test configures to drive the local-vs-remote branch under test. */
function mockFetchImplementation(isLocalhost: boolean, authMode: "local" | "jwt" = "jwt") {
return vi.fn().mockImplementation((url: string, init?: RequestInit) => {
const path = url.replace("http://localhost:8000", "");
const ok = (json: unknown, status = 200) =>
Promise.resolve({ ok: true, status, json: async () => json });
if (path === "/api/v1/auth/me") return ok(ADMIN_USER);
if (path === "/api/v1/system/status") return ok(systemStatusBody(isLocalhost, authMode));
if (path === "/api/v1/notification-destinations") return ok([]);
if (path === "/api/v1/companies") return ok([]);
if (path === "/api/v1/system/logs") return ok([]);
if (path === "/api/v1/user-api-keys") return ok([]);
if (path === "/api/v1/system/secrets") return ok([]);
if (path === "/api/v1/auth/security-events") return ok([]);
if (path === "/api/v1/admin/ip-bans") return ok([]);
if (path === "/api/v1/admin/unban-requests") return ok([]);
if (path === "/api/v1/db-viewer/session" && init?.method === "POST") {
return ok({ token: "mock-bootstrap-token" });
}
return Promise.resolve({ ok: false, status: 404, json: async () => ({ detail: "not found" }) });
});
}
beforeEach(() => {
vi.stubEnv("NEXT_PUBLIC_DB_VIEWER_URL", "https://db.ciagent.org");
});
describe("Settings - Database viewer box", () => {
it("links straight to the local Adminer instance for the local-dev bypass account", async () => {
vi.stubGlobal("fetch", mockFetchImplementation(true, "local"));
renderWithQueryClient(<SettingsPage />);
const link = await screen.findByRole("link", { name: /open database viewer/i });
expect(link).toHaveAttribute("href", "http://localhost:8081/?pgsql=postgres&username=ciagent");
expect(link).toHaveAttribute("target", "_blank");
});
it("mints a session token and opens the configured remote viewer URL for a real admin", async () => {
vi.stubGlobal("fetch", mockFetchImplementation(false, "jwt"));
const openSpy = vi.spyOn(window, "open").mockImplementation(() => null);
const user = userEvent.setup();
renderWithQueryClient(<SettingsPage />);
const button = await screen.findByRole("button", { name: /open database viewer/i });
await user.click(button);
await waitFor(() => {
expect(openSpy).toHaveBeenCalledWith(
"https://db.ciagent.org/_auth?token=mock-bootstrap-token",
"_blank",
"noopener,noreferrer",
);
});
});
it("shows a not-configured message when NEXT_PUBLIC_DB_VIEWER_URL is unset for a remote admin", async () => {
vi.stubEnv("NEXT_PUBLIC_DB_VIEWER_URL", "");
vi.stubGlobal("fetch", mockFetchImplementation(false, "jwt"));
renderWithQueryClient(<SettingsPage />);
expect(await screen.findByText(/not configured for this deployment/i)).toBeInTheDocument();
expect(screen.queryByRole("button", { name: /open database viewer/i })).not.toBeInTheDocument();
});
});
@@ -0,0 +1,148 @@
import { screen, waitFor } from "@testing-library/react";
import userEvent from "@testing-library/user-event";
import { beforeEach, describe, expect, it, vi } from "vitest";
import SettingsPage from "@/app/(app)/settings/page";
import { renderWithQueryClient } from "./test-utils";
const replaceMock = vi.fn();
vi.mock("next/navigation", () => ({
useRouter: () => ({ push: vi.fn(), replace: replaceMock }),
}));
const REGULAR_USER = {
id: "user-1",
email: "[email protected]",
display_name: "Regular User",
timezone: "America/New_York",
is_active: true,
is_admin: false,
auth_mode: "jwt",
must_change_password: false,
};
const LOCAL_DEV_USER = {
id: "00000000-0000-0000-0000-000000000001",
email: "[email protected]",
display_name: "Local Developer",
timezone: "America/New_York",
is_active: true,
is_admin: true,
auth_mode: "local",
must_change_password: false,
};
function systemStatusBody(isLocalhost: boolean, authMode: "local" | "jwt") {
return {
app_env: "development",
auth_mode: authMode,
llm_provider: "mock",
search_provider: "mock",
sms_enabled: false,
sms_provider: "twilio",
ninjapear_configured: false,
ninjapear_credit_balance: null,
ninjapear_estimated_credits_per_company: null,
is_localhost: isLocalhost,
turnstile_site_key: null,
components: [],
};
}
function mockFetchImplementation(
isLocalhost: boolean,
authMode: "local" | "jwt",
meUser: typeof REGULAR_USER,
) {
return vi.fn().mockImplementation((url: string, init?: RequestInit) => {
const path = url.replace("http://localhost:8000", "");
const ok = (json: unknown, status = 200) =>
Promise.resolve({ ok: true, status, json: async () => json });
if (path === "/api/v1/auth/me") return ok(meUser);
if (path === "/api/v1/system/status") return ok(systemStatusBody(isLocalhost, authMode));
if (path === "/api/v1/notification-destinations") return ok([]);
if (path === "/api/v1/companies") return ok([]);
if (path === "/api/v1/system/logs") return ok([]);
if (path === "/api/v1/user-api-keys") return ok([]);
if (path === "/api/v1/system/secrets") return ok([]);
if (path === "/api/v1/auth/security-events") return ok([]);
if (path === "/api/v1/admin/ip-bans") return ok([]);
if (path === "/api/v1/admin/unban-requests") return ok([]);
if (path === "/api/v1/auth/me" && init?.method === "DELETE") return ok(undefined, 204);
return Promise.resolve({ ok: false, status: 404, json: async () => ({ detail: "not found" }) });
});
}
beforeEach(() => {
replaceMock.mockClear();
});
describe("Settings - Delete account box", () => {
it("renders for a regular (non-local-dev) user, disabled until a password is typed", async () => {
vi.stubGlobal("fetch", mockFetchImplementation(false, "jwt", REGULAR_USER));
renderWithQueryClient(<SettingsPage />);
const button = await screen.findByRole("button", { name: /delete account/i });
expect(button).toBeDisabled();
const user = userEvent.setup();
await user.type(screen.getByLabelText(/confirm your password/i), "correct-horse-1");
expect(button).not.toBeDisabled();
});
it("is hidden entirely for the local-dev bypass account", async () => {
vi.stubGlobal("fetch", mockFetchImplementation(true, "local", LOCAL_DEV_USER));
renderWithQueryClient(<SettingsPage />);
await screen.findByText(LOCAL_DEV_USER.email);
expect(screen.queryByRole("button", { name: /delete account/i })).not.toBeInTheDocument();
});
it("submits the password, then clears tokens and redirects home on success", async () => {
const fetchMock = vi.fn().mockImplementation((url: string, init?: RequestInit) => {
const path = url.replace("http://localhost:8000", "");
if (path === "/api/v1/auth/me" && init?.method === "DELETE") {
expect(JSON.parse(init.body as string)).toEqual({ password: "correct-horse-1" });
return Promise.resolve({ ok: true, status: 204, json: async () => undefined });
}
return mockFetchImplementation(false, "jwt", REGULAR_USER)(url, init);
});
vi.stubGlobal("fetch", fetchMock);
const user = userEvent.setup();
renderWithQueryClient(<SettingsPage />);
const button = await screen.findByRole("button", { name: /delete account/i });
await user.type(screen.getByLabelText(/confirm your password/i), "correct-horse-1");
await user.click(button);
await waitFor(() => {
expect(replaceMock).toHaveBeenCalledWith("/");
});
});
it("shows an error message when the password is wrong", async () => {
const fetchMock = vi.fn().mockImplementation((url: string, init?: RequestInit) => {
const path = url.replace("http://localhost:8000", "");
if (path === "/api/v1/auth/me" && init?.method === "DELETE") {
return Promise.resolve({
ok: false,
status: 401,
json: async () => ({ detail: "Incorrect password" }),
});
}
return mockFetchImplementation(false, "jwt", REGULAR_USER)(url, init);
});
vi.stubGlobal("fetch", fetchMock);
const user = userEvent.setup();
renderWithQueryClient(<SettingsPage />);
const button = await screen.findByRole("button", { name: /delete account/i });
await user.type(screen.getByLabelText(/confirm your password/i), "wrong-password");
await user.click(button);
expect(await screen.findByText(/incorrect password/i)).toBeInTheDocument();
});
});
+15
View File
@@ -106,6 +106,7 @@ services:
args: args:
NEXT_PUBLIC_API_URL: ${NEXT_PUBLIC_API_URL} NEXT_PUBLIC_API_URL: ${NEXT_PUBLIC_API_URL}
NEXT_PUBLIC_GIT_REPO_URL: ${NEXT_PUBLIC_GIT_REPO_URL:-} NEXT_PUBLIC_GIT_REPO_URL: ${NEXT_PUBLIC_GIT_REPO_URL:-}
NEXT_PUBLIC_DB_VIEWER_URL: ${NEXT_PUBLIC_DB_VIEWER_URL:-}
restart: unless-stopped restart: unless-stopped
depends_on: depends_on:
- api - api
@@ -150,6 +151,19 @@ services:
- "2222:22" - "2222:22"
logging: *default-logging logging: *default-logging
# Settings -> Database viewer: internal-only, reached through nginx's
# db.ciagent.org block (app/api/v1/db_viewer.py issues the credential -
# see that module's docstring). No host port published.
adminer:
image: adminer:4.8.1-standalone
restart: unless-stopped
environment:
ADMINER_DEFAULT_SERVER: postgres
depends_on:
postgres:
condition: service_healthy
logging: *default-logging
nginx: nginx:
image: nginx:1.27-alpine image: nginx:1.27-alpine
restart: unless-stopped restart: unless-stopped
@@ -157,6 +171,7 @@ services:
- web - web
- api - api
- gitea - gitea
- adminer
volumes: volumes:
- ./infrastructure/nginx/nginx.conf:/etc/nginx/nginx.conf:ro - ./infrastructure/nginx/nginx.conf:/etc/nginx/nginx.conf:ro
# Cloudflare Origin CA cert/key, generated once via the Cloudflare # Cloudflare Origin CA cert/key, generated once via the Cloudflare
+13
View File
@@ -110,5 +110,18 @@ services:
depends_on: depends_on:
- api - api
adminer:
image: adminer:4.8.1-standalone
restart: unless-stopped
environment:
ADMINER_DEFAULT_SERVER: postgres
ports:
# Loopback-only, matching this app's loopback-trust philosophy - not
# meant to be reachable from the LAN. See Settings -> Database.
- "127.0.0.1:8081:8080"
depends_on:
postgres:
condition: service_healthy
volumes: volumes:
postgres-data: postgres-data:
+3 -1
View File
@@ -17,8 +17,10 @@ COPY apps/web ./
# *browser* will use, not a Docker-internal service name. See .env.example. # *browser* will use, not a Docker-internal service name. See .env.example.
ARG NEXT_PUBLIC_API_URL ARG NEXT_PUBLIC_API_URL
ARG NEXT_PUBLIC_GIT_REPO_URL ARG NEXT_PUBLIC_GIT_REPO_URL
ARG NEXT_PUBLIC_DB_VIEWER_URL
ENV NEXT_PUBLIC_API_URL=${NEXT_PUBLIC_API_URL} \ ENV NEXT_PUBLIC_API_URL=${NEXT_PUBLIC_API_URL} \
NEXT_PUBLIC_GIT_REPO_URL=${NEXT_PUBLIC_GIT_REPO_URL} NEXT_PUBLIC_GIT_REPO_URL=${NEXT_PUBLIC_GIT_REPO_URL} \
NEXT_PUBLIC_DB_VIEWER_URL=${NEXT_PUBLIC_DB_VIEWER_URL}
RUN npm run build RUN npm run build
+52 -1
View File
@@ -28,7 +28,7 @@ http {
# HTTPS at the edge, but the origin shouldn't 400 a direct :80 probe. # HTTPS at the edge, but the origin shouldn't 400 a direct :80 probe.
server { server {
listen 80; listen 80;
server_name ciagent.org api.ciagent.org git.ciagent.org; server_name ciagent.org api.ciagent.org git.ciagent.org db.ciagent.org;
return 301 https://$host$request_uri; return 301 https://$host$request_uri;
} }
@@ -59,6 +59,12 @@ http {
proxy_set_header Host $host; proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# NinjaPear's funding endpoint is documented as taking up to 5
# minutes (see app/enrichment/ninjapear.py's _FUNDING_TIMEOUT) -
# the enrichment-section-refresh endpoint calls it synchronously,
# so nginx's 60s default would otherwise 504 before it finishes.
proxy_read_timeout 320s;
proxy_send_timeout 320s;
} }
} }
@@ -79,4 +85,49 @@ http {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
} }
} }
# Settings -> Database viewer (Adminer). Access isn't gated by a shared
# password - the app itself (app/api/v1/db_viewer.py) mints a short-lived
# token from a live admin session, which /_auth here exchanges for a
# session cookie that /_verify re-checks (including a fresh `is_admin`
# lookup) on every request via auth_request. See DEPLOYMENT.md.
server {
listen 443 ssl;
server_name db.ciagent.org;
ssl_certificate /etc/nginx/certs/cloudflare-origin.pem;
ssl_certificate_key /etc/nginx/certs/cloudflare-origin.key;
# Adminer renders a raw SQL/data editor - not meant to be framed or
# MIME-sniffed as another content type.
add_header X-Frame-Options "DENY" always;
add_header X-Content-Type-Options "nosniff" always;
# Optional extra hardening for admins with a static IP: uncomment
# and set your own address to additionally require it alongside a
# valid session (default `satisfy all` - both must pass, not either).
# allow 203.0.113.9;
# deny all;
location = /_verify {
internal;
proxy_pass http://api:8000/api/v1/db-viewer/verify;
proxy_pass_request_body off;
proxy_set_header Content-Length "";
proxy_set_header Cookie $http_cookie;
}
location = /_auth {
proxy_pass http://api:8000/api/v1/db-viewer/bootstrap;
proxy_set_header Host $host;
}
location / {
auth_request /_verify;
proxy_pass http://adminer:8080;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
} }
Regular → Executable
+10
View File
@@ -28,4 +28,14 @@ docker compose -f docker-compose.prod.yml build api worker beat web
docker compose -f docker-compose.prod.yml run --rm api alembic upgrade head docker compose -f docker-compose.prod.yml run --rm api alembic upgrade head
docker compose -f docker-compose.prod.yml up -d docker compose -f docker-compose.prod.yml up -d
# nginx's proxy_pass resolves the api/web service names to a container IP
# once, at its own worker-process startup - `up -d` above only recreates
# the containers whose image/config actually changed, so nginx (unchanged)
# keeps running with the OLD ip, now pointing at a dead container.
# Confirmed live: this caused a real multi-hour outage (every request to
# api.ciagent.org / ciagent.org 502'd) after the containers it proxies to
# got rebuilt out from under it. `restart` forces new worker processes,
# which re-resolve the current IPs via Docker's embedded DNS.
docker compose -f docker-compose.prod.yml restart nginx
echo "$(date -Iseconds) deploy complete: $REMOTE" echo "$(date -Iseconds) deploy complete: $REMOTE"
Regular → Executable
View File