PocketVeto/ARCHITECTURE.md

# PocketVeto — Architecture & Feature Documentation

> **App brand:** PocketVeto
> **Repo:** civicstack
> **Purpose:** Citizen-grade US Congress monitoring with AI-powered bill analysis, per-claim citations, and personalized tracking.

---

## Table of Contents

1. [Overview](#overview)
2. [Tech Stack](#tech-stack)
3. [Infrastructure & Docker](#infrastructure--docker)
4. [Configuration & Environment](#configuration--environment)
5. [Database Schema](#database-schema)
6. [Alembic Migrations](#alembic-migrations)
7. [Backend API](#backend-api)
8. [Celery Workers & Pipeline](#celery-workers--pipeline)
9. [LLM Service](#llm-service)
10. [Frontend](#frontend)
11. [Authentication](#authentication)
12. [Key Architectural Patterns](#key-architectural-patterns)
13. [Feature History](#feature-history)
14. [Deployment](#deployment)

---

## Overview

PocketVeto is a self-hosted, full-stack application that automatically tracks US Congress legislation, fetches bill text, generates AI summaries with per-claim source citations, correlates bills with news and Google Trends, and presents everything through a personalized dashboard. Users follow bills, members of Congress, and policy topics; the system surfaces relevant activity in their feed.

```
Congress.gov API → Poller → DB → Document Fetcher → GovInfo
                                                       ↓
                                                 LLM Processor
                                                       ↓
                                                  BillBrief
                                                 (cited AI brief)
                                                       ↓
                                              News Fetcher + Trend Scorer
                                                       ↓
                                               Next.js Frontend
```

---

## Tech Stack

| Layer | Technology |
|---|---|
| Reverse Proxy | Nginx (alpine) |
| Backend API | FastAPI + SQLAlchemy (async) |
| Task Queue | Celery 5 + Redis |
| Task Scheduler | Celery Beat + RedBeat (Redis-backed) |
| Database | PostgreSQL 16 |
| Cache / Broker | Redis 7 |
| Frontend | Next.js 15, React, Tailwind CSS, TypeScript |
| Auth | JWT (python-jose) + bcrypt (passlib) |
| LLM | Multi-provider factory: OpenAI, Anthropic, Gemini, Ollama |
| Bill Metadata | Congress.gov API (api.data.gov key) |
| Bill Text | GovInfo API (same api.data.gov key) |
| News | NewsAPI.org (100 req/day free tier) |
| Trends | Google Trends via pytrends |

---

## Infrastructure & Docker

### Services (`docker-compose.yml`)

```
postgres:16-alpine
  DB:       pocketveto
  User:     congress
  Port:     5432 (internal)

redis:7-alpine
  Port:     6379 (internal)
  Role:     Celery broker, result backend, RedBeat schedule store

api  (civicstack-api image)
  Port:     8000 (internal)
  Command:  alembic upgrade head && uvicorn app.main:app --host 0.0.0.0 --port 8000
  Depends:  postgres (healthy), redis (healthy)

worker  (civicstack-worker image)
  Command:  celery -A app.workers.celery_app worker -Q polling,documents,llm,news -c 4
  Depends:  postgres (healthy), redis (healthy)

beat  (civicstack-beat image)
  Command:  celery -A app.workers.celery_app beat -S redbeat.RedBeatScheduler
  Depends:  redis (healthy)

frontend  (civicstack-frontend image)
  Port:     3000 (internal)
  Build:    Next.js standalone output

nginx:alpine
  Port:     80 → public
  Routes:   /api/* → api:8000 | /* → frontend:3000
```

### Nginx Config (`nginx/nginx.conf`)

- `resolver 127.0.0.11 valid=10s` — re-resolves Docker DNS after container restarts (prevents stale-IP 502s on redeploy)
- `/api/` → FastAPI, 120s read timeout
- `/_next/static/` → frontend with 1-day cache header
- `/` → frontend with WebSocket upgrade support

---

## Configuration & Environment

Copy `.env.example` → `.env` and fill in keys before first run.

```env
# Network
LOCAL_URL=http://localhost
PUBLIC_URL=                          # optional, e.g. https://yourapp.com

# Auth
JWT_SECRET_KEY=                      # python -c "import secrets; print(secrets.token_hex(32))"

# PostgreSQL
POSTGRES_USER=congress
POSTGRES_PASSWORD=congress
POSTGRES_DB=pocketveto

# Redis
REDIS_URL=redis://redis:6379/0

# Congress.gov + GovInfo (shared key from api.data.gov)
DATA_GOV_API_KEY=
CONGRESS_POLL_INTERVAL_MINUTES=30

# LLM — pick one provider
LLM_PROVIDER=openai                  # openai | anthropic | gemini | ollama
OPENAI_API_KEY=
OPENAI_MODEL=gpt-4o
ANTHROPIC_API_KEY=
ANTHROPIC_MODEL=claude-opus-4-6
GEMINI_API_KEY=
GEMINI_MODEL=gemini-2.0-flash
OLLAMA_BASE_URL=http://host.docker.internal:11434
OLLAMA_MODEL=llama3.1

# News & Trends
NEWSAPI_KEY=
PYTRENDS_ENABLED=true
```

**Runtime overrides:** LLM provider/model and poll interval can be changed live through the Admin page — stored in the `app_settings` table and take precedence over env vars.

---

## Database Schema

### `bills`
Primary key: `bill_id` — natural key in format `{congress}-{type}-{number}` (e.g. `119-hr-1234`).

| Column | Type | Notes |
|---|---|---|
| bill_id | varchar (PK) | |
| congress_number | int | |
| bill_type | varchar | `hr`, `s`, `hjres`, `sjres` (tracked); `hres`, `sres`, `hconres`, `sconres` (not tracked) |
| bill_number | int | |
| title | text | |
| short_title | text | |
| sponsor_id | varchar (FK → members) | bioguide_id |
| introduced_date | date | |
| latest_action_date | date | |
| latest_action_text | text | |
| status | varchar | |
| chamber | varchar | House / Senate |
| congress_url | varchar | congress.gov link |
| govtrack_url | varchar | |
| last_checked_at | timestamptz | |
| actions_fetched_at | timestamptz | |
| created_at / updated_at | timestamptz | |

Indexes: `congress_number`, `latest_action_date`, `introduced_date`, `chamber`, `sponsor_id`

---

### `bill_actions`

| Column | Type | Notes |
|---|---|---|
| id | int (PK) | |
| bill_id | varchar (FK → bills, CASCADE) | |
| action_date | date | |
| action_text | text | |
| action_type | varchar | |
| chamber | varchar | |
| created_at | timestamptz | |

---

### `bill_documents`
Stores fetched bill text versions from GovInfo.

| Column | Type | Notes |
|---|---|---|
| id | int (PK) | |
| bill_id | varchar (FK → bills, CASCADE) | |
| doc_type | varchar | `bill_text`, `committee_report`, `amendment` |
| doc_version | varchar | Introduced, Enrolled, etc. |
| govinfo_url | varchar | Source URL on GovInfo |
| raw_text | text | Full extracted text |
| fetched_at | timestamptz | |
| created_at | timestamptz | |

---

### `bill_briefs`
AI-generated analysis. `key_points` and `risks` are JSONB arrays of cited objects.

| Column | Type | Notes |
|---|---|---|
| id | int (PK) | |
| bill_id | varchar (FK → bills, CASCADE) | |
| document_id | int (FK → bill_documents, SET NULL) | |
| brief_type | varchar | `full` (first version) or `amendment` (diff from prior version) |
| summary | text | 2-4 paragraph plain-language summary |
| key_points | jsonb | `[{text, citation, quote}]` |
| risks | jsonb | `[{text, citation, quote}]` |
| deadlines | jsonb | `[{date, description}]` |
| topic_tags | jsonb | `["healthcare", "taxation", ...]` |
| llm_provider | varchar | Which provider generated this brief |
| llm_model | varchar | Specific model name |
| govinfo_url | varchar (nullable) | Source document URL (from bill_documents) |
| created_at | timestamptz | |

Indexes: `bill_id`, `topic_tags` (GIN for JSONB containment queries)

**Citation structure** — each `key_points`/`risks` item:
```json
{
  "text": "The bill allocates $50B for defense",
  "citation": "Section 301(a)(2)",
  "quote": "There is hereby appropriated for fiscal year 2026, $50,000,000,000 for the Department of Defense...",
  "label": "cited_fact"
}
```
`label` is `"cited_fact"` when the claim is explicitly stated in the quoted text, or `"inference"` when it is an analytical interpretation. Old briefs without this field render without a badge (backward compatible).

---

### `members`
Primary key: `bioguide_id` (Congress.gov canonical identifier).

| Column | Type | Notes |
|---|---|---|
| bioguide_id | varchar (PK) | |
| name | varchar | Stored as "Last, First" |
| first_name / last_name | varchar | |
| party | varchar | |
| state | varchar | |
| chamber | varchar | |
| district | varchar (nullable) | House only |
| photo_url | varchar (nullable) | |
| official_url | varchar (nullable) | Member's official website |
| congress_url | varchar (nullable) | congress.gov profile link |
| birth_year | varchar(10) (nullable) | |
| address | varchar (nullable) | DC office address |
| phone | varchar(50) (nullable) | DC office phone |
| terms_json | json (nullable) | Array of `{congress, startYear, endYear, chamber, partyName, stateName, district}` |
| leadership_json | json (nullable) | Array of `{type, congress, current}` |
| sponsored_count | int (nullable) | Total bills sponsored (lifetime) |
| cosponsored_count | int (nullable) | Total bills cosponsored (lifetime) |
| detail_fetched | timestamptz (nullable) | Set when bio detail was enriched from Congress.gov |
| created_at / updated_at | timestamptz | |

Member detail fields (`congress_url` through `detail_fetched`) are populated lazily on first profile view via a Congress.gov detail API call. The `detail_fetched` timestamp is the gate for scheduling member interest scoring.

### `member_trend_scores`
One record per member per day (mirrors `trend_scores` for bills).

| Column | Type | Notes |
|---|---|---|
| id | int (PK) | |
| member_id | varchar (FK → members, CASCADE) | bioguide_id |
| score_date | date | |
| newsapi_count | int | Articles from NewsAPI (30-day window) |
| gnews_count | int | Articles from Google News RSS |
| gtrends_score | float | Google Trends interest 0–100 |
| composite_score | float | Weighted combination 0–100 (same formula as bill trend scores) |

Unique constraint: `(member_id, score_date)`. Indexes: `member_id`, `score_date`, `composite_score`.

### `member_news_articles`
News articles correlated to a specific member of Congress.

| Column | Type | Notes |
|---|---|---|
| id | int (PK) | |
| member_id | varchar (FK → members, CASCADE) | bioguide_id |
| source | varchar | News outlet |
| headline | text | |
| url | varchar | Unique per `(member_id, url)` |
| published_at | timestamptz | |
| relevance_score | float | Default 1.0 |
| created_at | timestamptz | |

---

### `users`

| Column | Type | Notes |
|---|---|---|
| id | int (PK) | |
| email | varchar (unique) | |
| hashed_password | varchar | bcrypt |
| is_admin | bool | First registered user = true |
| notification_prefs | jsonb | ntfy topic URL, ntfy auth method/token/credentials, ntfy enabled, RSS token, quiet_hours_start/end (0–23 local), timezone (IANA name, e.g. `America/New_York`), alert_filters (nested dict: `{neutral: {...}, pocket_veto: {...}, pocket_boost: {...}}` — 8 boolean keys per mode) |
| rss_token | varchar (nullable) | Unique token for personal RSS feed URL |
| created_at | timestamptz | |

---

### `follows`

| Column | Type | Notes |
|---|---|---|
| id | int (PK) | |
| user_id | int (FK → users, CASCADE) | |
| follow_type | varchar | `bill`, `member`, `topic` |
| follow_value | varchar | bill_id, bioguide_id, or topic name |
| follow_mode | varchar | `neutral` \| `pocket_veto` \| `pocket_boost` (default `neutral`) |
| created_at | timestamptz | |

Unique constraint: `(user_id, follow_type, follow_value)`

---

### `bill_notes`
One private note per user per bill. Stored in the app; never shared.

| Column | Type | Notes |
|---|---|---|
| id | int (PK) | |
| user_id | int (FK → users, CASCADE) | |
| bill_id | varchar (FK → bills, CASCADE) | |
| content | text | User's note text |
| pinned | bool | If true, note floats above `BriefPanel` on the detail page |
| created_at | timestamptz | |
| updated_at | timestamptz | |

Unique constraint: `(user_id, bill_id)` — one note per user per bill. PUT endpoint upserts (create or update).

---

### `news_articles`

| Column | Type | Notes |
|---|---|---|
| id | int (PK) | |
| bill_id | varchar (FK → bills, CASCADE) | |
| source | varchar | News outlet |
| headline | varchar | |
| url | varchar | Unique per `(bill_id, url)` — same article can appear across multiple bills |
| published_at | timestamptz | |
| relevance_score | float | Default 1.0 |
| created_at | timestamptz | |

---

### `trend_scores`
One record per bill per day.

| Column | Type | Notes |
|---|---|---|
| id | int (PK) | |
| bill_id | varchar (FK → bills, CASCADE) | |
| score_date | date | |
| newsapi_count | int | Articles from NewsAPI (30-day window) |
| gnews_count | int | Articles from Google News RSS |
| gtrends_score | float | Google Trends interest 0–100 |
| composite_score | float | Weighted combination 0–100 |
| created_at | timestamptz | |

**Composite score formula:**
```
newsapi_pts  = min(newsapi_count / 20,  1.0) × 40   # saturates at 20 articles
gnews_pts    = min(gnews_count  / 50,  1.0) × 30   # saturates at 50 articles
gtrends_pts  = (gtrends_score / 100)        × 30
composite    = newsapi_pts + gnews_pts + gtrends_pts   # range 0–100
```

---

### `committees` / `committee_bills`

| committees | committee_id (PK), name, chamber, type |
|---|---|
| committee_bills | id, committee_id (FK), bill_id (FK), referred_date |

---

### `member_scores`
Nightly-computed effectiveness scores for members of Congress.

| Column | Type | Notes |
|---|---|---|
| id | int (PK) | |
| member_id | varchar (FK → members, CASCADE) | bioguide_id |
| score_date | date | |
| sponsored_count | int | Bills sponsored in current congress |
| advanced_count | int | Sponsored bills that advanced at least one stage |
| cosponsored_count | int | Bills cosponsored |
| enacted_count | int | Sponsored bills enacted into law |
| composite_score | float | Weighted effectiveness score 0–100 |
| calculated_at | timestamptz | |

Unique constraint: `(member_id, score_date)`. Nightly Celery task `calculate_member_effectiveness_scores` populates this table.

---

### `app_settings`
Key-value store for runtime-configurable settings.

| Key | Purpose |
|---|---|
| `congress_last_polled_at` | ISO timestamp of last successful poll |
| `llm_provider` | Overrides `LLM_PROVIDER` env var |
| `llm_model` | Overrides provider default model |
| `congress_poll_interval_minutes` | Overrides env var |
| `llm_active_batch` | JSON blob tracking in-flight LLM batch jobs (OpenAI / Anthropic Batch API) |

---

### `notifications`
Stores notification events for dispatching to user channels (ntfy, RSS).

| Column | Type | Notes |
|---|---|---|
| id | int (PK) | |
| user_id | int (FK → users, CASCADE) | |
| bill_id | varchar (FK → bills, SET NULL) | nullable |
| event_type | varchar | `new_document`, `new_amendment`, `bill_updated`, `weekly_digest` |
| payload | jsonb | `{bill_title, bill_label, brief_summary, bill_url, action_category, milestone_tier}` |
| dispatched_at | timestamptz (nullable) | NULL = pending dispatch |
| created_at | timestamptz | |

`action_category` in payload (new events): one of `vote`, `presidential`, `committee_report`, `calendar`, `procedural`, `referral`. `milestone_tier` is retained for backwards compatibility (`"referral"` or `"progress"`). The dispatcher checks `notification_prefs.alert_filters[follow_mode][action_category]` to decide whether to send. `new_document` and `new_amendment` events are filtered by event type directly (not action_category).

---

### `collections`
Named, curated groups of bills. Shareable via UUID token.

| Column | Type | Notes |
|---|---|---|
| id | int (PK) | |
| user_id | int (FK → users, CASCADE) | |
| name | varchar | 1–100 characters |
| slug | varchar | URL-safe version of name |
| is_public | bool | Signals inclusion in future public directory |
| share_token | uuid | Unique share URL token — read-only for non-owners |
| created_at | timestamptz | |

---

### `collection_bills`
Join table linking bills to collections.

| Column | Type | Notes |
|---|---|---|
| collection_id | int (FK → collections, CASCADE) | |
| bill_id | varchar (FK → bills, CASCADE) | |
| added_at | timestamptz | |

Unique constraint: `(collection_id, bill_id)`.

---

## Alembic Migrations

| File | Description |
|---|---|
| `0001_initial_schema.py` | All initial tables |
| `0002_widen_chamber_party_columns.py` | Wider varchar for Bill.chamber, Member.party |
| `0003_widen_member_state_district.py` | Wider varchar for Member.state, Member.district |
| `0004_add_brief_type.py` | BillBrief.brief_type column (`full`/`amendment`) |
| `0005_add_users_and_user_follows.py` | users table + user_id FK on follows; drops global follows |
| `0006_add_brief_govinfo_url.py` | BillBrief.govinfo_url for frontend source links |
| `0007_add_member_bio_fields.py` | Member extended bio: `congress_url`, `birth_year`, `address`, `phone`, `terms_json`, `leadership_json`, `sponsored_count`, `cosponsored_count`, `detail_fetched` |
| `0008_add_member_interest_tables.py` | New tables: `member_trend_scores`, `member_news_articles` |
| `0009_fix_news_articles_url_uniqueness.py` | Changed `news_articles.url` from globally unique to per-bill unique `(bill_id, url)` |
| `0010_backfill_bill_congress_urls.py` | Backfill congress_url on existing bill records |
| `0011_add_notifications.py` | `notifications` table + `rss_token` column on users |
| `0012_dedupe_bill_actions_unique.py` | Unique constraint on `(bill_id, action_date, action_text)` for idempotent action ingestion |
| `0013_add_follow_mode.py` | `follow_mode` column on `follows` (`neutral` / `pocket_veto` / `pocket_boost`) |
| `0014_add_bill_notes.py` | `bill_notes` table with unique constraint `(user_id, bill_id)` |
| `0015_add_collections.py` | `collections` table (`id`, `user_id`, `name`, `slug`, `is_public`, `share_token` UUID, `created_at`) |
| `0016_add_collection_bills.py` | `collection_bills` join table (`collection_id` FK, `bill_id` FK, `added_at`); `share_token` UUID column on `bill_briefs` |

Migrations run automatically on API startup: `alembic upgrade head`.

---

## Backend API

Base URL: `/api`
Auth header: `Authorization: Bearer <jwt>`

### `/api/auth`

| Method | Path | Auth | Description |
|---|---|---|---|
| POST | `/register` | — | Create account. First user → admin. Returns token + user. |
| POST | `/login` | — | Returns token + user. |
| GET | `/me` | Required | Current user info. |

### `/api/bills`

| Method | Path | Auth | Description |
|---|---|---|---|
| GET | `/` | — | Paginated bill list. Query: `chamber`, `topic`, `sponsor_id`, `q`, `page`, `per_page`, `sort`. Includes `has_document` flag per bill via a single batch query. |
| GET | `/{bill_id}` | — | Full bill detail with sponsor, actions, briefs, news, trend scores. |
| GET | `/{bill_id}/actions` | — | Action timeline, newest first. |
| GET | `/{bill_id}/news` | — | Related news articles, limit 20. |
| GET | `/{bill_id}/trend` | — | Trend score history. Query: `days` (7–365, default 30). |
| POST | `/{bill_id}/draft-letter` | — | Generate a constituent letter draft via the configured LLM. Body: `{stance, recipient, tone, selected_points, include_citations, zip_code?}`. Returns `{draft: string}`. ZIP code is used in the prompt only — never stored or logged. |

### `/api/members`

| Method | Path | Auth | Description |
|---|---|---|---|
| GET | `/` | — | Paginated members. Query: `chamber`, `party`, `state`, `q`, `page`, `per_page`. |
| GET | `/{bioguide_id}` | — | Member detail. On first view, lazily enriches bio from Congress.gov and queues member interest scoring. Returns `latest_trend` if scored. |
| GET | `/{bioguide_id}/bills` | — | Member's sponsored bills, paginated. |
| GET | `/{bioguide_id}/trend` | — | Member trend score history. Query: `days` (7–365, default 30). |
| GET | `/{bioguide_id}/news` | — | Member's recent news articles, limit 20. |

### `/api/follows`

| Method | Path | Auth | Description |
|---|---|---|---|
| GET | `/` | Required | Current user's follows. |
| POST | `/` | Required | Add follow `{follow_type, follow_value}`. Idempotent. |
| DELETE | `/{id}` | Required | Remove follow (ownership checked). |

### `/api/dashboard`

| Method | Path | Auth | Description |
|---|---|---|---|
| GET | `/` | Required | Personalized feed from followed bills/members/topics + trending. Returns `{feed, trending, follows}`. |

### `/api/search`

| Method | Path | Auth | Description |
|---|---|---|---|
| GET | `/` | — | Full-text search. Query: `q` (min 2 chars). Returns `{bills, members}`. |

### `/api/settings`

| Method | Path | Auth | Description |
|---|---|---|---|
| GET | `/` | Required | Current settings (DB overrides env). |
| PUT | `/` | Admin | Update `{key, value}`. Allowed keys: `llm_provider`, `llm_model`, `congress_poll_interval_minutes`. |
| POST | `/test-llm` | Admin | Test LLM connection with a lightweight ping (max_tokens=20). Returns `{status, provider, model, reply}`. |
| GET | `/llm-models?provider=X` | Admin | Fetch available models from the live provider API. Supports openai, anthropic, gemini, ollama. |

### `/api/notes`

| Method | Path | Auth | Description |
|---|---|---|---|
| GET | `/{bill_id}` | Required | Get user's note for a bill. Returns 404 if no note exists. |
| PUT | `/{bill_id}` | Required | Upsert note `{content, pinned}`. Creates or updates (one note per user per bill). |
| DELETE | `/{bill_id}` | Required | Delete the user's note. Returns 404 if none exists. |

### `/api/notifications`

| Method | Path | Auth | Description |
|---|---|---|---|
| GET | `/settings` | Required | User's notification preferences (ntfy URL/auth, quiet hours, timezone, RSS token, digest settings). |
| PUT | `/settings` | Required | Update notification preferences. |
| POST | `/settings/rss-reset` | Required | Regenerate RSS token (invalidates old URL). |
| GET | `/feed/{rss_token}.xml` | — | Personal RSS feed of notification events for this user. |
| POST | `/test/ntfy` | Required | Send a test ntfy push using the provided credentials (not saved). |
| POST | `/test/rss` | Required | Generate a test RSS entry and return event count. |
| POST | `/test/follow-mode` | Required | Simulate a follow-mode notification to preview delivery behavior. |
| GET | `/history` | Required | Recent notification events (dispatched + pending). |
| GET | `/unsubscribe/{token}` | — | One-click email unsubscribe; invalidates token and disables email for the user. |

### `/api/collections`

| Method | Path | Auth | Description |
|---|---|---|---|
| GET | `/` | Required | Current user's collections with bill count. |
| POST | `/` | Required | Create collection `{name, is_public}`. Generates slug + share_token. |
| GET | `/{id}` | Required | Collection detail with bills. |
| PUT | `/{id}` | Required | Update `{name, is_public}`. |
| DELETE | `/{id}` | Required | Delete collection (owner only). |
| POST | `/{id}/bills/{bill_id}` | Required | Add bill to collection. |
| DELETE | `/{id}/bills/{bill_id}` | Required | Remove bill from collection. |
| GET | `/share/{token}` | — | Public read-only view of a collection by share token. |

### `/api/share`

| Method | Path | Auth | Description |
|---|---|---|---|
| GET | `/brief/{token}` | — | Public brief + bill data by share token (from `bill_briefs.share_token`). |
| GET | `/collection/{token}` | — | Public collection + bills by share token. |

### `/api/admin`

| Method | Path | Auth | Description |
|---|---|---|---|
| GET | `/users` | Admin | All users with follow counts. |
| DELETE | `/users/{id}` | Admin | Delete user (cannot delete self). Cascades follows. |
| PATCH | `/users/{id}/toggle-admin` | Admin | Promote/demote admin status (cannot change self). |
| GET | `/stats` | Admin | Pipeline counters: total bills, docs fetched, briefs generated, pending LLM, missing metadata/sponsors/actions, uncited briefs, unlabeled briefs (cited objects without a fact/inference label). |
| GET | `/api-health` | Admin | Test each external API in parallel; returns status + latency for Congress.gov, GovInfo, NewsAPI, Google News. |
| POST | `/trigger-poll` | Admin | Queue immediate Congress.gov poll. |
| POST | `/trigger-member-sync` | Admin | Queue member sync. |
| POST | `/trigger-trend-scores` | Admin | Queue trend score calculation. |
| POST | `/trigger-fetch-actions` | Admin | Queue action fetch for recently active bills (last 30 days). |
| POST | `/backfill-all-actions` | Admin | Queue action fetch for ALL bills with no action history (one-time catch-up). |
| POST | `/backfill-sponsors` | Admin | Queue one-off task to populate `sponsor_id` on bills where it is NULL. |
| POST | `/backfill-metadata` | Admin | Fill null `introduced_date`, `chamber`, `congress_url` by re-fetching bill detail. |
| POST | `/backfill-citations` | Admin | Delete pre-citation briefs and re-queue LLM using stored document text. |
| POST | `/backfill-labels` | Admin | Classify existing cited brief points as `cited_fact` or `inference` in-place — one compact LLM call per brief, no re-generation. |
| POST | `/resume-analysis` | Admin | Re-queue LLM for docs with no brief; re-queue doc fetch for bills with no doc. |
| POST | `/trigger-weekly-digest` | Admin | Manually trigger the weekly digest task for all users now. |
| POST | `/bills/{bill_id}/reprocess` | Admin | Queue document + action fetches for a specific bill (debugging). |
| GET | `/task-status/{task_id}` | Admin | Celery task status and result. |

### `/api/alignment`

| Method | Path | Auth | Description |
|---|---|---|---|
| GET | `/` | Required | Representation Alignment View — for each followed member, compares their roll-call vote positions against the user's followed topics. Returns alignment breakdown per member. Neutral presentation — no scorecard framing. |

### `/api/health`

| Method | Path | Description |
|---|---|---|
| GET | `/` | Simple health check `{status: "ok", timestamp}`. |
| GET | `/detailed` | Tests PostgreSQL + Redis. Returns per-service status. |

---

## Celery Workers & Pipeline

**Celery app name:** `pocketveto`
**Broker / Backend:** Redis

### Queue Routing

| Queue | Workers | Tasks |
|---|---|---|
| `polling` | worker | `app.workers.congress_poller.*`, `app.workers.notification_dispatcher.*` |
| `documents` | worker | `fetch_bill_documents` |
| `llm` | worker | `process_document_with_llm` |
| `news` | worker | `app.workers.news_fetcher.*`, `app.workers.trend_scorer.*`, `app.workers.member_interest.*` |

**Worker settings:**
- `task_acks_late = True` — task removed from queue only after completion, not on pickup
- `worker_prefetch_multiplier = 1` — prevents workers from hoarding LLM tasks
- Serialization: JSON

### Beat Schedule (RedBeat, stored in Redis)

| Schedule | Task | When |
|---|---|---|
| Configurable (default 30 min) | `poll_congress_bills` | Continuous |
| Every 6 hours | `fetch_news_for_active_bills` | Ongoing |
| Daily 2 AM UTC | `calculate_all_trend_scores` | Nightly |
| Every 12 hours (at :30) | `fetch_news_for_active_members` | Ongoing |
| Daily 3 AM UTC | `calculate_all_member_trend_scores` | Nightly |
| Daily 4 AM UTC | `fetch_actions_for_active_bills` | Nightly |
| Daily 5 AM UTC | `calculate_member_effectiveness_scores` | Nightly |
| Every 5 minutes | `dispatch_notifications` | Continuous |
| Every 10 minutes | `poll_llm_batch_results` | Continuous (when a batch is active) |
| Mondays 8:30 AM UTC | `send_weekly_digest` | Weekly |

---

### Pipeline Flow

```
1. congress_poller.poll_congress_bills()
   ↳ Fetches bills updated since last poll (fromDateTime param)
   ↳ Filters: only hr, s, hjres, sjres (legislation that can become law)
   ↳ First run: seeds from 60 days back
   ↳ New bills: fetches bill detail endpoint to get sponsor (list endpoint
     has no sponsor data), upserts Member, sets bill.sponsor_id
   ↳ New bills → fetch_bill_documents.delay(bill_id)
   ↳ Updated bills → fetch_bill_documents.delay(bill_id) if changed
   ↳ Updated bills → emit bill_updated notification if categorize_action() returns a category:
       - vote: passed, failed, agreed to, roll call
       - presidential: signed, vetoed, enacted, presented to the president
       - committee_report: markup, ordered to be reported, ordered reported, reported by, discharged
       - calendar: placed on
       - procedural: cloture, conference
       - referral: referred to
       → All three follow types (bill, sponsor, topic) receive notification.
         Whether it is dispatched depends on the user's per-mode alert_filters in notification_prefs.

2. document_fetcher.fetch_bill_documents(bill_id)
   ↳ Gets text versions from Congress.gov (XML preferred, falls back to HTML/PDF)
   ↳ Fetches raw text from GovInfo
   ↳ Idempotent: skips if doc_version already stored
   ↳ Stores BillDocument with govinfo_url + raw_text
   ↳ → process_document_with_llm.delay(document_id)

3. llm_processor.process_document_with_llm(document_id)
   ↳ Rate limited: 10/minute
   ↳ Idempotent: skips if brief exists for document
   ↳ Determines type:
       - No prior brief → "full" brief
       - Prior brief exists → "amendment" brief (diff vs previous)
   ↳ Calls configured LLM provider
   ↳ Stores BillBrief with cited key_points and risks
   ↳ → fetch_news_for_bill.delay(bill_id)

4. news_fetcher.fetch_news_for_bill(bill_id)
   ↳ Queries NewsAPI + Google News RSS using bill title/number
   ↳ Deduplicates by (bill_id, url) — same article can appear for multiple bills
   ↳ Stores NewsArticle records

5. trend_scorer.calculate_all_trend_scores()  [nightly]
   ↳ Bills active in last 90 days
   ↳ Skips bills already scored today
   ↳ Fetches: NewsAPI count + Google News RSS count + Google Trends score
   ↳ Calculates composite_score (0–100)
   ↳ Stores TrendScore record

Member interest pipeline (independent of bill pipeline):

6. member_interest.fetch_member_news(bioguide_id)  [on first profile view + every 12h]
   ↳ Triggered on first member profile view (non-blocking via .delay())
   ↳ Queries NewsAPI + Google News RSS using member name + title
   ↳ Deduplicates by (member_id, url)
   ↳ Stores MemberNewsArticle records

7. member_interest.calculate_member_trend_score(bioguide_id)  [on first profile view + nightly]
   ↳ Triggered on first member profile view (non-blocking via .delay())
   ↳ Only runs if member detail has been fetched (gate: detail_fetched IS NOT NULL)
   ↳ Fetches: NewsAPI count + Google News RSS count + Google Trends score
   ↳ Uses the same composite formula as bills
   ↳ Stores MemberTrendScore record
```

---

## LLM Service

**File:** `backend/app/services/llm_service.py`

### Provider Factory

```python
get_llm_provider() → LLMProvider
```

Reads `LLM_PROVIDER` from AppSetting (DB) then env var. Instantiates the matching provider class.

| Provider | Class | Key Setting |
|---|---|---|
| `openai` | `OpenAIProvider` | `OPENAI_API_KEY`, `OPENAI_MODEL` |
| `anthropic` | `AnthropicProvider` | `ANTHROPIC_API_KEY`, `ANTHROPIC_MODEL` |
| `gemini` | `GeminiProvider` | `GEMINI_API_KEY`, `GEMINI_MODEL` |
| `ollama` | `OllamaProvider` | `OLLAMA_BASE_URL`, `OLLAMA_MODEL` |

All providers implement:
```python
generate_brief(doc_text, bill_metadata) → ReverseBrief
generate_amendment_brief(new_text, prev_text, bill_metadata) → ReverseBrief
generate_text(prompt) → str   # free-form text, used by draft letter generator
```

### ReverseBrief Dataclass

```python
@dataclass
class ReverseBrief:
    summary: str
    key_points: list[dict]   # [{text, citation, quote, label}]
    risks: list[dict]        # [{text, citation, quote, label}]
    deadlines: list[dict]    # [{date, description}]
    topic_tags: list[str]
    llm_provider: str
    llm_model: str
```

### Prompt Design

**Full brief prompt** instructs the LLM to produce:
```json
{
  "summary": "2-4 paragraph plain-language explanation",
  "key_points": [
    {
      "text": "claim",
      "citation": "Section X(y)",
      "quote": "verbatim excerpt ≤80 words",
      "label": "cited_fact"
    }
  ],
  "risks": [
    {
      "text": "concern",
      "citation": "Section X(y)",
      "quote": "verbatim excerpt ≤80 words",
      "label": "inference"
    }
  ],
  "deadlines": [{"date": "YYYY-MM-DD or null", "description": "..."}],
  "topic_tags": ["healthcare", "taxation"]
}
```

`label` classification rules baked into the system prompt: `"cited_fact"` if the claim is explicitly stated in the quoted text; `"inference"` if it is an analytical interpretation, projection, or implication not literally stated. The UI shows a neutral "Inferred" badge on inference items only (cited_fact is the clean default).

**Amendment brief prompt** focuses on what changed between document versions.

### LLM Batch API

OpenAI and Anthropic support async batch endpoints that process requests at ~50% of the standard per-token cost, with results delivered within 24 hours.

- `submit_llm_batch` Celery task — collects pending documents, submits a batch request to the provider's batch endpoint, and stores the batch ID + document mapping in `AppSetting("llm_active_batch")`.
- `poll_llm_batch_results` Celery task — runs every 10 minutes; checks batch status; on completion, writes `BillBrief` records from the returned results and clears the active batch state.
- Batch mode is opt-in per provider. When active, `process_document_with_llm` routes jobs to the batch queue instead of calling the real-time API directly.
- State key `llm_active_batch` in `app_settings` is a JSON blob: `{provider, batch_id, document_ids: [...]}`.

**Smart truncation:** Bills exceeding the token budget are trimmed — 75% of budget from the start (preamble/purpose), 25% from the end (enforcement/effective dates), with an omission notice in the middle.

**Token budgets:**
- OpenAI / Anthropic / Gemini: 6,000 tokens
- Ollama: 3,000 tokens (local models have smaller context windows)

---

## Frontend

**Framework:** Next.js 15 (App Router), TypeScript, Tailwind CSS
**State:** Zustand (auth), TanStack Query (server state)
**HTTP:** Axios with JWT interceptor

### Pages

| Route | Description |
|---|---|
| `/` | Dashboard — personalized feed + trending bills |
| `/bills` | Browse all bills with search, chamber/topic filters, pagination |
| `/bills/[id]` | Bill detail — tabbed UI (Analysis, Timeline, Votes, Notes); AI brief with § citations, action timeline, roll-call votes, draft letter, personal notes |
| `/members` | Browse members of Congress, filter by chamber/party/state |
| `/members/[id]` | Member profile — bio, contact info, leadership roles, service history, sponsored bills, public interest trend chart, recent news |
| `/following` | User's followed bills, members, and topics with accordion sections and topic filters |
| `/topics` | Browse and follow policy topics |
| `/collections` | User's collections (watchlists) — create, manage, share |
| `/collections/[id]` | Collection detail — bills in the collection, share link |
| `/notifications` | Notification settings — ntfy config, alert filters (per follow mode), quiet hours, digest mode, RSS |
| `/how-it-works` | Feature guide covering follow modes, collections, notifications, AI briefs, bill browsing |
| `/settings` | Admin panel (admin only) |
| `/login` | Email + password sign-in |
| `/register` | Account creation |
| `/share/brief/[token]` | Public shareable brief view — no sidebar, no auth required |
| `/share/collection/[token]` | Public shareable collection view — no sidebar, no auth required |

### Key Components

**`BriefPanel.tsx`**
Orchestrates AI brief display. If the latest brief is type `amendment`, shows an amber "What Changed" badge. Renders the latest brief via `AIBriefCard`. Below it, a collapsible "Version History" lists all older briefs; clicking one expands an inline `AIBriefCard`.

**`AIBriefCard.tsx`**
Renders the LLM brief. For cited items (new format), shows a `§ Section X(y)` chip next to each bullet. Clicking the chip expands an inline panel with:
- Blockquoted verbatim excerpt from the bill
- "View source →" link to GovInfo (opens in new tab)
- One chip open at a time per card
- Inference items show a neutral "Inferred" badge (analytical interpretation, not a literal quote)
- Old plain-string briefs render without chips (graceful backward compat)

**`ActionTimeline.tsx`**
Renders the legislative action history as a vertical timeline. Accepts optional `latestActionDate`/`latestActionText` fallback props — when `actions` is empty but a latest action exists (actions not yet fetched from Congress.gov), shows a single "latest known action" entry with a note that full history loads in the background.

**`MobileHeader.tsx`**
Top bar shown only on mobile (`md:hidden`). Displays the PocketVeto logo and a hamburger button that opens the slide-in drawer.

**`AuthGuard.tsx`**
Client component wrapping the entire app. Waits for Zustand hydration, then redirects unauthenticated users to `/login`. Public paths (`/login`, `/register`) bypass the guard. Implements the responsive shell: desktop sidebar always-visible (`hidden md:flex`), mobile drawer with backdrop overlay controlled by `drawerOpen` state.

**`Sidebar.tsx`**
Navigation with: Home, Bills, Members, Following, Topics, Settings (admin only). Shows current user email + logout button at the bottom. Accepts optional `onClose` prop — when provided (mobile drawer context), renders an X close button in the header and calls `onClose` on every nav link click.

**`DraftLetterPanel.tsx`**
Collapsible panel rendered below `BriefPanel` on the bill detail page (only when a brief exists). Lets users select up to 3 cited points from the brief, choose stance (YES/NO), tone (short/polite/firm), and optionally enter a ZIP code (not stored). Stance auto-populates from the user's follow mode (`pocket_boost` → YES, `pocket_veto` → NO); clears if they unfollow. Recipient (house/senate) is derived from the bill's chamber. Calls `POST /{bill_id}/draft-letter` and renders the plain-text draft in a readonly textarea with a copy-to-clipboard button.

**`NotesPanel.tsx`**
Collapsible private-note panel on the bill detail page (auth-gated; returns null for guests). Header shows "My Note" with last-saved date and a pin icon when a note exists. Expanded: auto-resize textarea, Pin/Unpin toggle, Save button (disabled when clean), and Trash icon (only when note exists). Pinned notes render above `BriefPanel`; unpinned notes render below `DraftLetterPanel`. Uses `retry: false, throwOnError: false` on the TanStack Query so a 404 (no note yet) is treated as empty state rather than an error. Mutations show a "Saved!" flash for 2 seconds.

**`BillCard.tsx`**
Compact bill preview showing bill ID, title, sponsor with party badge, latest action date, status, and a text availability indicator: `Brief` (green, analysis done) / `Pending` (amber, text retrieved but not yet analysed) / `No text` (muted, nothing published on Congress.gov).

**`TrendChart.tsx`**
Line chart of `composite_score` over time with tooltip breakdown of each data source.

**`WelcomeBanner.tsx`**
Dismissible onboarding card rendered at the top of the dashboard. Shown only to guests (no JWT token). On dismiss — via the × button, the "Dismiss" link, or the "Browse Bills" CTA — sets `localStorage["pv_seen_welcome"] = "1"` and hides permanently. Reads localStorage after mount to avoid hydration mismatch; renders nothing until client-side state is resolved.

### Topic Tags (`lib/topics.ts`)

Shared constant `KNOWN_TOPICS: string[]` — the canonical list of 20 policy topics used across the app. Topic tag pills on the bill detail page and bill listing pages are filtered to this list, preventing spurious LLM-generated tags from appearing as clickable filters.

```typescript
export const KNOWN_TOPICS = [
  "agriculture", "budget", "defense", "education", "energy",
  "environment", "finance", "foreign-policy", "healthcare", "housing",
  "immigration", "infrastructure", "judiciary", "labor", "science",
  "social-security", "taxation", "technology", "trade", "veterans"
]
```

### Utility Functions (`lib/utils.ts`)

```typescript
partyBadgeColor(party) → Tailwind classes
  "Republican" → "bg-red-600 text-white"
  "Democrat"   → "bg-blue-600 text-white"
  other        → "bg-slate-500 text-white"

chamberBadgeColor(chamber) → Tailwind badge classes
  "Senate"  → amber/gold  (bg-amber-100 text-amber-700 …)
  "House"   → slate/silver (bg-slate-100 text-slate-600 …)

partyColor(party) → text color class (used inline)
trendColor(score) → color class based on score thresholds
billLabel(type, number) → "H.R. 1234", "S. 567", etc.
formatDate(date) → "Feb 28, 2026"
```

### Auth Store (`stores/authStore.ts`)

```typescript
interface AuthState {
  token: string | null
  user: { id: number; email: string; is_admin: boolean } | null
  setAuth(token, user): void
  logout(): void
}
// Persisted to localStorage as "pocketveto-auth"
```

**Auth-aware query keys:** TanStack Query keys that return different data for guests vs authenticated users include `!!token` in their key (e.g. `["dashboard", !!token]`). This ensures a fresh fetch fires automatically on login or logout without manual cache invalidation.

---

## Authentication

- **Algorithm:** HS256 JWT, 7-day expiry
- **Storage:** Zustand store persisted to `localStorage` key `pocketveto-auth`
- **Injection:** Axios request interceptor reads from localStorage and adds `Authorization: Bearer <token>` to every request
- **First user:** The first account registered automatically receives `is_admin = true`
- **Admin role:** Required for PUT/POST `/api/settings`, all `/api/admin/*` endpoints, and viewing the Settings page in the UI
- **No email verification:** Accounts are active immediately on registration
- **Public endpoints:** `/api/bills`, `/api/members`, `/api/search`, `/api/health` — no auth required

---

## Key Architectural Patterns

### Idempotent Workers
Every Celery task checks for existing records before processing. Combined with `task_acks_late=True`, this means:
- Tasks can be retried without creating duplicates
- Worker crashes don't lose work (task stays in queue until acknowledged)

### Incremental Polling
The Congress.gov poller uses `fromDateTime` to fetch only recently updated bills, tracking the last poll timestamp in `app_settings`. On first run it seeds 60 days back to avoid processing thousands of old bills.

### Bill Type Filtering
Only tracks legislation that can become law:
- `hr` (House Resolution → Bill)
- `s` (Senate Bill)
- `hjres` (House Joint Resolution)
- `sjres` (Senate Joint Resolution)

Excluded (procedural, cannot become law): `hres`, `sres`, `hconres`, `sconres`

### Queue Specialization
Separate queues prevent a flood of LLM tasks from blocking time-sensitive polling tasks. Worker prefetch of 1 prevents any single worker from hoarding slow LLM jobs.

### LLM Provider Abstraction
All LLM providers implement the same interface. Switching providers is a single admin setting change — no code changes, no restart required (the factory reads from DB on each task invocation).

### JSONB for Flexible Brief Storage
`key_points`, `risks`, `deadlines`, `topic_tags` are stored as JSONB. This means schema changes (adding `citation`/`quote` in v0.2.0, adding `label` in v0.6.0) required no migrations — only the LLM prompt and application code changed. Old string-format briefs, cited-object briefs without labels, and fully-labelled briefs all coexist in the same column and render correctly at each fidelity level.

### Redis-backed Beat Schedule (RedBeat)
The Celery Beat schedule is stored in Redis rather than in memory. This means the beat scheduler can restart without losing schedule state or double-firing tasks.

### Encrypted Sensitive Preferences
ntfy passwords (basic auth credentials stored in `notification_prefs` JSONB) are encrypted at rest using Fernet symmetric encryption (`app/core/crypto.py`). The encryption key is read from `ENCRYPTION_SECRET_KEY` env var. Fernet uses AES-128-CBC with HMAC-SHA256 authentication. The key must be generated once (`Fernet.generate_key()`) and must not change after data is written — re-encryption is required if the key is rotated. All other notification prefs (ntfy topic URL, token auth, quiet hours, etc.) are stored in plaintext.

### Docker DNS Re-resolution
Nginx uses `resolver 127.0.0.11 valid=10s` (Docker's internal DNS) so upstream container IPs are refreshed every 10 seconds. Without this, nginx caches the IP at startup and returns 502 errors after any container is recreated.

---

## Feature History

### v0.1.0 — Foundation
- Docker Compose stack: PostgreSQL, Redis, FastAPI, Celery, Next.js, Nginx
- Congress.gov API integration: bill polling, member sync
- GovInfo document fetching with intelligent truncation
- Multi-provider LLM service (OpenAI, Anthropic, Gemini, Ollama)
- AI brief generation: summary, key points, risks, deadlines, topic tags
- Amendment-aware processing: diffs new bill versions against prior
- NewsAPI + Google News RSS article correlation
- Google Trends (pytrends) scoring
- Composite trend score (0–100) with weighted formula
- Full-text bill search (PostgreSQL tsvector)
- Member of Congress browsing
- Global follows (bill / member / topic)
- Personalized dashboard feed
- Admin settings page (LLM provider selection, data source status)
- Manual Celery task triggers from UI
- Bill type filtering: only legislation that can become law
- 60-day seed window on fresh install

**Multi-User Auth (added to v0.1.0):**
- Email + password registration/login (JWT, bcrypt)
- Per-user follow scoping
- Admin role (first user = admin)
- Admin user management: list, delete, promote/demote
- AuthGuard with login/register pages
- Analysis status dashboard (auto-refresh every 30s)

### v0.3.0 — Member Profiles & Mobile UI

**Member Interest Tracking:**
- `member_trend_scores` and `member_news_articles` tables (migration 0008)
- `member_interest` Celery worker: `fetch_member_news`, `calculate_member_trend_score`, `fetch_news_for_active_members`, `calculate_all_member_trend_scores`
- Member interest scoring uses the identical composite formula as bills (NewsAPI + GNews + pytrends)
- New beat schedules: member news every 12h, member trend scores nightly at 3 AM UTC
- Lazy enrichment: on first profile view, bio is fetched from Congress.gov detail API and interest scoring is queued non-blocking
- Member detail fields added: `congress_url`, `birth_year`, `address`, `phone`, `terms_json`, `leadership_json`, `sponsored_count`, `cosponsored_count`, `detail_fetched` (migration 0007)
- New API endpoints: `GET /api/members/{id}/trend` and `GET /api/members/{id}/news`
- Member detail page redesigned: photo, bio header with party/state/district/birth year, contact info (address, phone, website, congress.gov), current leadership badges, trend chart ("Public Interest"), news panel, legislation stats (sponsored/cosponsored counts), full service history timeline, all leadership roles history

**News Deduplication Fix:**
- `news_articles.url` changed from globally unique to per-bill unique `(bill_id, url)` (migration 0009)
- The same article can now appear in multiple bills' news panels
- `fetch_news_for_bill` now fetches from both NewsAPI and Google News RSS (previously GNews was volume-signal only)

**Mobile UI:**
- `MobileHeader.tsx` — hamburger + logo top bar, hidden on desktop (`md:hidden`)
- `AuthGuard.tsx` — responsive shell: desktop sidebar always-on, mobile slide-in drawer with backdrop
- `Sidebar.tsx` — `onClose` prop for drawer mode (X button + close on nav click)
- Dashboard grid: `grid-cols-1 md:grid-cols-3` (single column on mobile)
- Members page: `grid-cols-1 sm:grid-cols-2` (single column on mobile, two on tablet+)
- Topics page: `grid-cols-1 sm:grid-cols-2`

### v0.4.0 — Notifications, Admin Health Panel, Bill Action Pipeline

**Notifications (Phase 1 complete):**
- `notifications` table — stores events per user (new_brief, bill_updated, new_action)
- ntfy dispatch — Celery task POSTs to user's ntfy topic URL (self-hosted or ntfy.sh); optional auth token
- RSS feed — tokenized per-user XML feed at `/api/notifications/feed/{token}.xml`
- `dispatch_notifications` beat task — runs every 5 minutes, fans out unsent events to enabled channels
- Notification settings UI — ntfy topic URL, auth token, enable/disable, RSS URL with copy button

**Bill Action Pipeline:**
- `fetch_bill_actions` Celery task — fetches full legislative history from Congress.gov, idempotent on `(bill_id, action_date, action_text)`, updates `Bill.actions_fetched_at`
- `fetch_actions_for_active_bills` nightly batch — queues action fetches for bills active in last 30 days
- `backfill_all_bill_actions` — one-time task to fetch actions for all bills with `actions_fetched_at IS NULL`
- Beat schedule entry at 4 AM UTC
- `ActionTimeline` updated: shows full history when fetched; falls back to `latest_action_date`/`latest_action_text` with "latest known action" label when history not yet loaded

**"What Changed" — BriefPanel:**
- New `BriefPanel.tsx` component wrapping `AIBriefCard`
- When latest brief is type `amendment`: shows amber "What Changed" badge row + date
- Collapsible "Version History" section listing older briefs (date, type badge, truncated summary)
- Clicking a history row expands an inline `AIBriefCard` for that version

**LLM Provider Improvements:**
- Live model picker — `GET /api/settings/llm-models?provider=X` fetches available models from each provider's API (OpenAI SDK, Anthropic REST, Gemini SDK, Ollama tags endpoint)
- DB overrides now fully propagated: `get_llm_provider(provider, model)` accepts explicit params; all call sites read from `app_settings`
- Default Gemini model updated: `gemini-1.5-pro` (deprecated) → `gemini-2.0-flash`
- Test connection replaced with lightweight ping (max_tokens=20, 3-word prompt) instead of full brief generation

**Admin Panel Overhaul:**
- Bill Pipeline section: progress bar + breakdown table (total, text published, no text yet, AI briefs, pending LLM, uncited)
- External API Health: Run Tests button, parallel health checks for Congress.gov / GovInfo / NewsAPI / Google News RSS with latency display
- Manual Controls redesigned as health panel: each action has a status dot (green/red/gray), description, contextual count badge (e.g. "⚠ 12 bills missing metadata"), and Run button
- Task status polling: after triggering a task, button shows spinning icon; polls `/api/admin/task-status/{id}` every 5s; shows task ID prefix + completion/failure state
- New stat fields: `bills_missing_sponsor`, `bills_missing_metadata`, `bills_missing_actions`, `pending_llm`, `no_text_bills`
- New admin tasks: Backfill Dates & Links, Backfill All Action Histories, Resume Analysis

**Chamber Color Badges:**
- `chamberBadgeColor(chamber)` utility: amber/gold for Senate, slate/silver for House
- Applied everywhere chamber is displayed: BillCard, bill detail header

**Bill Detail Page:**
- "No bill text published" state — shown when `has_document=false` and no briefs; includes bill label, date, and congress.gov link
- `has_document` field added to `BillDetailSchema` and `BillDetail` TypeScript type
- `introduced_date` shown conditionally (not rendered when null, preventing "Introduced: —")
- Admin reprocess endpoint: `POST /api/admin/bills/{bill_id}/reprocess`

### v0.5.0 — Follow Modes, Public Browsing & Draft Letter Generator

**Follow Modes:**
- `follow_mode` column on `follows` table: `neutral | pocket_veto | pocket_boost`
- `FollowButton` replaced with a mode-selector dropdown (shield/zap/heart icons, descriptions for each mode)
- `pocket_veto` — alert only on advancement milestones; `pocket_boost` — all changes + action prompts
- Mode stored per-follow; respected by notification dispatcher

**Public Browsing:**
- Unauthenticated guests can browse bills, members, topics, and the trending dashboard
- `AuthModal` gates follow and other interactive actions
- Sidebar and nav adapt to guest state (no email/logout shown)
- All public endpoints already auth-free; guard refactored to allow guest reads

**Draft Constituent Letter Generator (email_gen):**
- `DraftLetterPanel.tsx` — collapsible UI below `BriefPanel` for bills with a brief
- User selects up to 3 cited points from the brief, picks stance (YES/NO), tone, optional ZIP (not stored)
- Stance pre-fills from follow mode; clears on unfollow (ref-tracked, not effect-guarded)
- Recipient derived from bill chamber — no dropdown needed
- `POST /api/bills/{bill_id}/draft-letter` endpoint: reads LLM provider/model from `AppSetting` (respects Settings page), wraps LLM errors with human-readable messages (quota, rate limit, auth)
- `generate_text(prompt) → str` added to `LLMProvider` ABC and all four providers

**Bill Text Status Indicators:**
- `has_document` field added to `BillSchema` (list endpoint) via a single batch `SELECT DISTINCT` — no per-card queries
- `BillCard` shows: `Brief` (green) / `Pending` (amber) / `No text` (muted) based on brief + document state

### v0.6.0 — Phase 2: Change-driven Alerts & Fact/Inference Labeling

**Change-driven Alerts:**
- `notification_utils.py` milestone keyword list expanded: added `"markup"` (markup sessions) and `"conference"` (conference committee)
- New `is_referral_action()` classifier for committee referrals (`"referred to"`)
- Two-tier notification system: `milestone_tier` field in `NotificationEvent.payload`
  - `"progress"` — high-signal milestones (passed, signed, markup, etc.): all follow types notified
  - `"referral"` — committee referral: pocket_veto and pocket_boost notified; neutral silently dropped
- **Topic followers now receive `bill_updated` milestone notifications** — previously they only received `new_document`/`new_amendment` events. Fixed by querying the bill's latest brief for `topic_tags` inside `_update_bill_if_changed()`
- All three follow types (bill, sponsor, topic) covered for both tiers

**Fact vs Inference Labeling:**
- `label: "cited_fact" | "inference"` added to every cited key_point and risk in the LLM JSON schema
- System prompt updated for all four providers (OpenAI, Anthropic, Gemini, Ollama)
- UI: neutral "Inferred" badge shown next to inference items in `AIBriefCard`; cited_fact items render cleanly without a badge
- `backfill_brief_labels` Celery task: classifies existing cited points in-place — one compact LLM call per brief (all points batched), updates JSONB with `flag_modified`, no brief re-generation
- `POST /api/admin/backfill-labels` endpoint + "Backfill Fact/Inference Labels" button in Admin panel
- `unlabeled_briefs` counter added to `/api/admin/stats` and pipeline breakdown table

**Admin Panel Cleanup:**
- Manual Controls split into two sections: always-visible recurring controls (Poll, Members, Trends, Actions, Resume) and a collapsible **Maintenance** section for one-time backfill tasks
- Maintenance section header shows "⚠ action needed" when any backfill has a non-zero count

### v0.8.0 — Personal Notes

**Private Per-Bill Notes:**
- `bill_notes` table (migration 0014) — `(user_id, bill_id)` unique constraint enforces one note per user per bill
- `BillNote` SQLAlchemy model with CASCADE relationships on both `User` and `Bill`
- `PUT /api/notes/{bill_id}` upsert — single endpoint handles create and update; no separate create route
- `GET /api/notes/{bill_id}` returns 404 when no note exists (not an error — treated as empty state by the frontend)
- `DELETE /api/notes/{bill_id}` removes the note; returns 404 if none
- All notes endpoints require authentication
- `NotesPanel.tsx` — collapsible UI; auth-gated (returns null for guests)
  - Auto-resize textarea via `scrollHeight`
  - Pin/Unpin toggle: pinned notes float above `BriefPanel`; unpinned render below `DraftLetterPanel`
  - `isDirty` check prevents saving unchanged content; "Saved!" flash for 2 seconds on success
  - `retry: false, throwOnError: false` so 404 = no note, not an error state
- Pin state fetched on page load via a shared `["note", billId]` query key (reused by both the page and `NotesPanel`)

### v0.7.0 — Weekly Digest & Local-Time Quiet Hours

**Weekly Digest:**
- `send_weekly_digest` Celery task — runs Mondays at 8:30 AM UTC
- Per user: queries bills updated in the past 7 days among followed items; sends a low-priority ntfy push listing up to 5 bills with brief summaries
- Creates a `NotificationEvent` with `event_type="weekly_digest"` and `dispatched_at=now` to prevent double-dispatch by the regular `dispatch_notifications` task; digest events still appear in the user's RSS feed
- Uses the most recently updated followed bill as `bill_id` anchor (FK is NOT NULL)
- `POST /api/admin/trigger-weekly-digest` endpoint + "Send Weekly Digest" button in Admin Manual Controls
- ntfy push: `Priority: low`, `Tags: newspaper,calendar`

**Local-Time Quiet Hours:**
- `quiet_hours_start` / `quiet_hours_end` are now interpreted in the user's **local time** rather than UTC
- Browser auto-detects timezone via `Intl.DateTimeFormat().resolvedOptions().timeZone` (IANA name) on the Notifications settings page
- Timezone saved in `notification_prefs` JSONB as `timezone` (no migration needed — JSONB is schema-flexible)
- Backend `_in_quiet_hours()` converts the dispatch UTC timestamp to local time using `zoneinfo.ZoneInfo` (Python 3.9+ stdlib); falls back to UTC if no timezone stored
- `NotificationSettingsResponse` / `NotificationSettingsUpdate` schemas now include `timezone: Optional[str]`
- Notifications UI: quiet hours displayed in 12-hour AM/PM format; shows `"Times are in your local timezone: <IANA name>"` hint; amber warning if saved timezone differs from currently detected one (e.g., after travel)

### v0.6.1 — Welcome Banner & Dashboard Auth Fix

**Welcome Banner:**
- `WelcomeBanner.tsx` — dismissible onboarding card shown only to guests at the top of the dashboard
- Three bullet points: follow bills/members/topics, see what changed, Back to Source citations
- "Browse Bills" CTA navigates to `/bills` and dismisses; × and "Dismiss" button also dismiss
- Dismissed state stored in `localStorage["pv_seen_welcome"]`; never shown to logged-in users

**Dashboard Auth-Aware Query Key:**
- `useDashboard` hook query key changed from `["dashboard"]` to `["dashboard", !!token]`
- Fixes stale cache issue where logging in showed the guest feed until a manual refresh

### v0.2.2 — Sponsor Linking & Search Fixes
- **Root cause fixed:** Congress.gov list API does not return sponsor data — only the detail endpoint does. Poller now calls the detail endpoint for each new bill to get the sponsor and populate `bill.sponsor_id`
- **Backfill task:** `backfill_sponsor_ids` Celery task + `/api/admin/backfill-sponsors` endpoint + "Backfill Sponsors" button in Admin UI — fixes existing bills with `NULL` sponsor_id (~10 req/sec, ~3 min for 1,600 bills)
- **Member name search:** members are stored as "Last, First" in the `name` column; search now also matches "First Last" order using PostgreSQL `split_part()` — applied to both the Members page and global search
- **Search spaces:** removed `.trim()` on search `onChange` handlers in Members and Bills pages that was eating spaces as you typed
- **Member bills 500 error:** `get_member_bills` endpoint now eagerly loads `Bill.sponsor` via `selectinload` to prevent Pydantic MissingGreenlet error during serialization

### v0.2.0 — Citations
- **Per-claim citations on AI briefs:** every key point and risk includes:
  - `citation` — section reference (e.g., "Section 301(a)(2)")
  - `quote` — verbatim excerpt ≤80 words from that section
- `§` citation chip UI on each bullet — click to expand quote + GovInfo source link
- `govinfo_url` stored on `BillBrief` for direct frontend access
- Old briefs (plain strings) render without chips — backward compatible
- Migration 0006: `govinfo_url` column on `bill_briefs`
- Party badges redesigned: solid `red-600` / `blue-600` / `slate-500` with white text, readable in both light and dark mode
- Tailwind content scan extended to include `lib/` directory
- Nginx DNS resolver fix: prevents stale-IP 502s after container restarts

### v0.9.0 — Collections, Shareable Links & Notification Improvements

**Collections / Watchlists:**
- `collections` and `collection_bills` tables (migrations 0015/0016)
- Named, curated bill groups with public/private flag and UUID share token
- `CollectionPicker` popover on bill detail page — create or add to existing collections from the bookmark icon
- `/collections` page — list, create, manage collections
- `/collections/[id]` — collection detail with share link
- `GET /api/collections/share/{token}` — public read-only collection view

**Shareable Brief Links:**
- `share_token` UUID column on `bill_briefs` (migration 0016)
- Share button (Share2 icon) in `BriefPanel` copies a public link
- `/api/share/brief/{token}` and `/api/share/collection/{token}` — public router, no auth
- `/share/brief/[token]` and `/share/collection/[token]` — public frontend pages, no sidebar
- `AuthGuard` updated: `/collections` → AUTH_REQUIRED; `/share/` prefix → NO_SHELL (uses `.startsWith()`)

**Notification Improvements:**
- Quiet hours dispatch fix: events held during quiet window fire correctly on next run
- Digest mode: bundled ntfy summary on daily or weekly schedule instead of per-event pushes
- `send_notification_digest` Celery task with `digest_frequency: "daily" | "weekly"` setting
- Notification history panel on settings page split into direct follows vs topic follows

### v0.9.3 — Granular Per-Mode Alert Filters

- `categorize_action()` replaces `is_milestone_action()` / `is_referral_action()` — maps action text to one of 6 named categories: `vote`, `presidential`, `committee_report`, `calendar`, `procedural`, `referral`
- `action_category` stored in every `NotificationEvent.payload` going forward; `milestone_tier` retained for RSS/history backward compatibility
- `alert_filters` added to `notification_prefs` JSONB — nested dict: `{neutral: {...}, pocket_veto: {...}, pocket_boost: {...}}`, each with 8 boolean keys covering all event types (`new_document`, `new_amendment`, and the 6 action categories)
- Dispatcher `_should_dispatch()` checks `prefs["alert_filters"][follow_mode][key]` — replaces two hardcoded per-mode suppression blocks
- Notifications settings page: tabbed **Alert Filters** section (Follow / Pocket Veto / Pocket Boost tabs), each with 8 independent toggles, Milestones parent checkbox (indeterminate-aware), **Load defaults** revert button, and per-tab **Save** button
- How It Works page updated with accurate per-mode default alert sets and filter customization guidance
- No DB migration required — `alert_filters` stored in existing JSONB column

### v0.9.5–v0.9.7 — Roll-Call Votes, Discovery Alerts & API Optimizations

- `bill_votes` and `member_vote_positions` tables; on-demand fetch from Congress.gov `/votes` endpoint
- `VotePanel` on bill detail shows yea/nay bar + followed member positions
- Discovery alert filters: member follow and topic follow events carry `source=member_follow|topic_follow`; filtered via `alert_filters.{source}.enabled` and per-category booleans
- Event payloads carry `follow_mode`, `matched_member_name`, `matched_member_id`, `matched_topic` for "why" reason lines in ntfy + notification history UI
- API optimizations: quota batching, ETag caching, async sponsor resolution

### v0.9.8 — LLM Batch API

- OpenAI and Anthropic async batch endpoints integrated; ~50% cost reduction for brief generation
- `submit_llm_batch` and `poll_llm_batch_results` Celery tasks
- Batch state stored in `AppSetting("llm_active_batch")` as a JSON blob
- Admin panel shows active batch status and allows manual batch submission/poll trigger

### v0.9.9–v0.9.10 — Accountability & Email Notifications

**Accountability (Phase 4):**
- Vote History surfaced in the bill action timeline; roll-call vote events tagged in the timeline view
- Member Effectiveness Score: nightly `calculate_member_effectiveness_scores` Celery task; transparent formula (sponsored, advanced, cosponsored, enacted); `member_scores` table; displayed on member profile with formula explanation
- Representation Alignment View: `GET /api/alignment` compares stanced follows vs member roll-call vote positions; neutral, informational presentation

**Email Notifications (v0.9.10):**
- SMTP integration using `smtplib`; port 465 = `SMTP_SSL`; port 587 = `STARTTLS`
- Resend-compatible (and generic SMTP relay)
- HTML email templates with bill summary, action details, and one-click unsubscribe token
- Per-user opt-in via notification settings UI
- Unsubscribe tokens are single-use, stored in the DB, and invalidated on use

**UX & Polish (v0.9.x):**
- Bill detail page refactored to a four-tab layout: Analysis, Timeline, Votes, Notes
- Topic tag pills on bill detail and listing pages — filtered to `KNOWN_TOPICS` from `frontend/lib/topics.ts`
- Collapsible sidebar: icon-only mode, collapse state persisted to `localStorage`
- Favicon: Landmark icon
- ntfy passwords encrypted at rest with Fernet (`app/core/crypto.py`); key from `ENCRYPTION_SECRET_KEY`

---

## Deployment

### First Deploy

```bash
cp .env.example .env
# Edit .env — add API keys, generate JWT_SECRET_KEY
docker compose up --build -d
```

Migrations run automatically. Navigate to the app, register the first account (it becomes admin).

### Updating

```bash
git pull origin main
docker compose up --build -d
docker compose exec nginx nginx -s reload   # if nginx wasn't recreated
```

### Useful Commands

```bash
# Check all service status
docker compose ps

# View logs
docker compose logs api --tail=50
docker compose logs worker --tail=50

# Force a bill poll now
# → Admin page → Manual Controls → Trigger Poll

# Check DB column layout
docker compose exec postgres psql -U congress -d pocketveto -c "\d bill_briefs"

# Tail live worker output
docker compose logs -f worker

# Restart a specific service
docker compose restart worker
```

### Bill Regeneration (Optional)

Existing briefs generated before v0.2.0 use plain strings (no citations). To regenerate with citations:

1. Delete existing `bill_briefs` rows (keeps `bill_documents` intact)
2. Re-queue all documents via a one-off script similar to `queue_docs.py`
3. Worker will regenerate using the new cited prompt at 10/minute
4. ~1,000 briefs ≈ 2 hours

This is **optional** — old string briefs render correctly in the UI with no citation chips.