feat(phase2): fact/inference labeling, change-driven alerts, admin cleanup

- Add label: cited_fact | inference to LLM brief schema (all 4 providers)
- Inferred badge in AIBriefCard for inference-labeled points
- backfill_brief_labels Celery task: classifies existing cited points in-place
- POST /api/admin/backfill-labels + unlabeled_briefs stat counter
- Expand milestone keywords: markup, conference
- Add is_referral_action() for committee referrals (referred to)
- Two-tier milestone notifications: progress tier (all follow modes) and
  referral tier (pocket_veto/boost only, neutral suppressed)
- Topic followers now receive bill_updated milestone notifications via
  latest brief topic_tags lookup in _update_bill_if_changed()
- Admin Manual Controls: collapsible Maintenance section for backfill tasks
- Update ARCHITECTURE.md and roadmap for Phase 2 completion

Co-Authored-By: Jack Levy
This commit is contained in:
Jack Levy
2026-03-01 17:34:45 -05:00
parent dc5e756749
commit 1e37c99599
12 changed files with 500 additions and 121 deletions

View File

@@ -238,9 +238,11 @@ Indexes: `bill_id`, `topic_tags` (GIN for JSONB containment queries)
{ {
"text": "The bill allocates $50B for defense", "text": "The bill allocates $50B for defense",
"citation": "Section 301(a)(2)", "citation": "Section 301(a)(2)",
"quote": "There is hereby appropriated for fiscal year 2026, $50,000,000,000 for the Department of Defense..." "quote": "There is hereby appropriated for fiscal year 2026, $50,000,000,000 for the Department of Defense...",
"label": "cited_fact"
} }
``` ```
`label` is `"cited_fact"` when the claim is explicitly stated in the quoted text, or `"inference"` when it is an analytical interpretation. Old briefs without this field render without a badge (backward compatible).
--- ---
@@ -324,6 +326,7 @@ News articles correlated to a specific member of Congress.
| user_id | int (FK → users, CASCADE) | | | user_id | int (FK → users, CASCADE) | |
| follow_type | varchar | `bill`, `member`, `topic` | | follow_type | varchar | `bill`, `member`, `topic` |
| follow_value | varchar | bill_id, bioguide_id, or topic name | | follow_value | varchar | bill_id, bioguide_id, or topic name |
| follow_mode | varchar | `neutral` \| `pocket_veto` \| `pocket_boost` (default `neutral`) |
| created_at | timestamptz | | | created_at | timestamptz | |
Unique constraint: `(user_id, follow_type, follow_value)` Unique constraint: `(user_id, follow_type, follow_value)`
@@ -397,12 +400,13 @@ Stores notification events for dispatching to user channels (ntfy, RSS).
| id | int (PK) | | | id | int (PK) | |
| user_id | int (FK → users, CASCADE) | | | user_id | int (FK → users, CASCADE) | |
| bill_id | varchar (FK → bills, SET NULL) | nullable | | bill_id | varchar (FK → bills, SET NULL) | nullable |
| event_type | varchar | e.g. `new_brief`, `bill_updated`, `new_action` | | event_type | varchar | `new_document`, `new_amendment`, `bill_updated` |
| headline | text | Short description for ntfy title | | payload | jsonb | `{bill_title, bill_label, brief_summary, bill_url, milestone_tier}` |
| body | text | Longer description for ntfy message / RSS content | | dispatched_at | timestamptz (nullable) | NULL = pending dispatch |
| dispatched_at | timestamptz (nullable) | NULL = not yet sent |
| created_at | timestamptz | | | created_at | timestamptz | |
`milestone_tier` in payload: `"progress"` (passed, signed, markup, conference, etc.) or `"referral"` (committee referral). Neutral follows silently skip referral-tier events; pocket_veto and pocket_boost receive them as early warnings.
--- ---
## Alembic Migrations ## Alembic Migrations
@@ -442,11 +446,12 @@ Auth header: `Authorization: Bearer <jwt>`
| Method | Path | Auth | Description | | Method | Path | Auth | Description |
|---|---|---|---| |---|---|---|---|
| GET | `/` | — | Paginated bill list. Query: `chamber`, `topic`, `sponsor_id`, `q`, `page`, `per_page`, `sort`. | | GET | `/` | — | Paginated bill list. Query: `chamber`, `topic`, `sponsor_id`, `q`, `page`, `per_page`, `sort`. Includes `has_document` flag per bill via a single batch query. |
| GET | `/{bill_id}` | — | Full bill detail with sponsor, actions, briefs, news, trend scores. | | GET | `/{bill_id}` | — | Full bill detail with sponsor, actions, briefs, news, trend scores. |
| GET | `/{bill_id}/actions` | — | Action timeline, newest first. | | GET | `/{bill_id}/actions` | — | Action timeline, newest first. |
| GET | `/{bill_id}/news` | — | Related news articles, limit 20. | | GET | `/{bill_id}/news` | — | Related news articles, limit 20. |
| GET | `/{bill_id}/trend` | — | Trend score history. Query: `days` (7365, default 30). | | GET | `/{bill_id}/trend` | — | Trend score history. Query: `days` (7365, default 30). |
| POST | `/{bill_id}/draft-letter` | — | Generate a constituent letter draft via the configured LLM. Body: `{stance, recipient, tone, selected_points, include_citations, zip_code?}`. Returns `{draft: string}`. ZIP code is used in the prompt only — never stored or logged. |
### `/api/members` ### `/api/members`
@@ -503,7 +508,7 @@ Auth header: `Authorization: Bearer <jwt>`
| GET | `/users` | Admin | All users with follow counts. | | GET | `/users` | Admin | All users with follow counts. |
| DELETE | `/users/{id}` | Admin | Delete user (cannot delete self). Cascades follows. | | DELETE | `/users/{id}` | Admin | Delete user (cannot delete self). Cascades follows. |
| PATCH | `/users/{id}/toggle-admin` | Admin | Promote/demote admin status (cannot change self). | | PATCH | `/users/{id}/toggle-admin` | Admin | Promote/demote admin status (cannot change self). |
| GET | `/stats` | Admin | Pipeline counters: total bills, docs fetched, briefs generated, pending LLM, missing metadata/sponsors/actions, uncited briefs. | | GET | `/stats` | Admin | Pipeline counters: total bills, docs fetched, briefs generated, pending LLM, missing metadata/sponsors/actions, uncited briefs, unlabeled briefs (cited objects without a fact/inference label). |
| GET | `/api-health` | Admin | Test each external API in parallel; returns status + latency for Congress.gov, GovInfo, NewsAPI, Google News. | | GET | `/api-health` | Admin | Test each external API in parallel; returns status + latency for Congress.gov, GovInfo, NewsAPI, Google News. |
| POST | `/trigger-poll` | Admin | Queue immediate Congress.gov poll. | | POST | `/trigger-poll` | Admin | Queue immediate Congress.gov poll. |
| POST | `/trigger-member-sync` | Admin | Queue member sync. | | POST | `/trigger-member-sync` | Admin | Queue member sync. |
@@ -513,6 +518,7 @@ Auth header: `Authorization: Bearer <jwt>`
| POST | `/backfill-sponsors` | Admin | Queue one-off task to populate `sponsor_id` on bills where it is NULL. | | POST | `/backfill-sponsors` | Admin | Queue one-off task to populate `sponsor_id` on bills where it is NULL. |
| POST | `/backfill-metadata` | Admin | Fill null `introduced_date`, `chamber`, `congress_url` by re-fetching bill detail. | | POST | `/backfill-metadata` | Admin | Fill null `introduced_date`, `chamber`, `congress_url` by re-fetching bill detail. |
| POST | `/backfill-citations` | Admin | Delete pre-citation briefs and re-queue LLM using stored document text. | | POST | `/backfill-citations` | Admin | Delete pre-citation briefs and re-queue LLM using stored document text. |
| POST | `/backfill-labels` | Admin | Classify existing cited brief points as `cited_fact` or `inference` in-place — one compact LLM call per brief, no re-generation. |
| POST | `/resume-analysis` | Admin | Re-queue LLM for docs with no brief; re-queue doc fetch for bills with no doc. | | POST | `/resume-analysis` | Admin | Re-queue LLM for docs with no brief; re-queue doc fetch for bills with no doc. |
| POST | `/bills/{bill_id}/reprocess` | Admin | Queue document + action fetches for a specific bill (debugging). | | POST | `/bills/{bill_id}/reprocess` | Admin | Queue document + action fetches for a specific bill (debugging). |
| GET | `/task-status/{task_id}` | Admin | Celery task status and result. | | GET | `/task-status/{task_id}` | Admin | Celery task status and result. |
@@ -570,6 +576,12 @@ Auth header: `Authorization: Bearer <jwt>`
has no sponsor data), upserts Member, sets bill.sponsor_id has no sponsor data), upserts Member, sets bill.sponsor_id
↳ New bills → fetch_bill_documents.delay(bill_id) ↳ New bills → fetch_bill_documents.delay(bill_id)
↳ Updated bills → fetch_bill_documents.delay(bill_id) if changed ↳ Updated bills → fetch_bill_documents.delay(bill_id) if changed
↳ Updated bills → emit bill_updated notification if action is a milestone:
- "progress" tier: passed/failed, signed/vetoed, enacted, markup, conference,
reported from committee, placed on calendar, cloture, roll call
→ all follow types (bill, sponsor, topic) receive notification
- "referral" tier: referred to committee
→ pocket_veto and pocket_boost only; neutral follows silently skip
2. document_fetcher.fetch_bill_documents(bill_id) 2. document_fetcher.fetch_bill_documents(bill_id)
↳ Gets text versions from Congress.gov (XML preferred, falls back to HTML/PDF) ↳ Gets text versions from Congress.gov (XML preferred, falls back to HTML/PDF)
@@ -641,6 +653,7 @@ All providers implement:
```python ```python
generate_brief(doc_text, bill_metadata) ReverseBrief generate_brief(doc_text, bill_metadata) ReverseBrief
generate_amendment_brief(new_text, prev_text, bill_metadata) ReverseBrief generate_amendment_brief(new_text, prev_text, bill_metadata) ReverseBrief
generate_text(prompt) str # free-form text, used by draft letter generator
``` ```
### ReverseBrief Dataclass ### ReverseBrief Dataclass
@@ -649,8 +662,8 @@ generate_amendment_brief(new_text, prev_text, bill_metadata) → ReverseBrief
@dataclass @dataclass
class ReverseBrief: class ReverseBrief:
summary: str summary: str
key_points: list[dict] # [{text, citation, quote}] key_points: list[dict] # [{text, citation, quote, label}]
risks: list[dict] # [{text, citation, quote}] risks: list[dict] # [{text, citation, quote, label}]
deadlines: list[dict] # [{date, description}] deadlines: list[dict] # [{date, description}]
topic_tags: list[str] topic_tags: list[str]
llm_provider: str llm_provider: str
@@ -664,16 +677,28 @@ class ReverseBrief:
{ {
"summary": "2-4 paragraph plain-language explanation", "summary": "2-4 paragraph plain-language explanation",
"key_points": [ "key_points": [
{"text": "claim", "citation": "Section X(y)", "quote": "verbatim excerpt ≤80 words"} {
"text": "claim",
"citation": "Section X(y)",
"quote": "verbatim excerpt ≤80 words",
"label": "cited_fact"
}
], ],
"risks": [ "risks": [
{"text": "concern", "citation": "Section X(y)", "quote": "verbatim excerpt ≤80 words"} {
"text": "concern",
"citation": "Section X(y)",
"quote": "verbatim excerpt ≤80 words",
"label": "inference"
}
], ],
"deadlines": [{"date": "YYYY-MM-DD or null", "description": "..."}], "deadlines": [{"date": "YYYY-MM-DD or null", "description": "..."}],
"topic_tags": ["healthcare", "taxation"] "topic_tags": ["healthcare", "taxation"]
} }
``` ```
`label` classification rules baked into the system prompt: `"cited_fact"` if the claim is explicitly stated in the quoted text; `"inference"` if it is an analytical interpretation, projection, or implication not literally stated. The UI shows a neutral "Inferred" badge on inference items only (cited_fact is the clean default).
**Amendment brief prompt** focuses on what changed between document versions. **Amendment brief prompt** focuses on what changed between document versions.
**Smart truncation:** Bills exceeding the token budget are trimmed — 75% of budget from the start (preamble/purpose), 25% from the end (enforcement/effective dates), with an omission notice in the middle. **Smart truncation:** Bills exceeding the token budget are trimmed — 75% of budget from the start (preamble/purpose), 25% from the end (enforcement/effective dates), with an omission notice in the middle.
@@ -715,6 +740,7 @@ Renders the LLM brief. For cited items (new format), shows a `§ Section X(y)` c
- Blockquoted verbatim excerpt from the bill - Blockquoted verbatim excerpt from the bill
- "View source →" link to GovInfo (opens in new tab) - "View source →" link to GovInfo (opens in new tab)
- One chip open at a time per card - One chip open at a time per card
- Inference items show a neutral "Inferred" badge (analytical interpretation, not a literal quote)
- Old plain-string briefs render without chips (graceful backward compat) - Old plain-string briefs render without chips (graceful backward compat)
**`ActionTimeline.tsx`** **`ActionTimeline.tsx`**
@@ -729,8 +755,11 @@ Client component wrapping the entire app. Waits for Zustand hydration, then redi
**`Sidebar.tsx`** **`Sidebar.tsx`**
Navigation with: Home, Bills, Members, Following, Topics, Settings (admin only). Shows current user email + logout button at the bottom. Accepts optional `onClose` prop — when provided (mobile drawer context), renders an X close button in the header and calls `onClose` on every nav link click. Navigation with: Home, Bills, Members, Following, Topics, Settings (admin only). Shows current user email + logout button at the bottom. Accepts optional `onClose` prop — when provided (mobile drawer context), renders an X close button in the header and calls `onClose` on every nav link click.
**`DraftLetterPanel.tsx`**
Collapsible panel rendered below `BriefPanel` on the bill detail page (only when a brief exists). Lets users select up to 3 cited points from the brief, choose stance (YES/NO), tone (short/polite/firm), and optionally enter a ZIP code (not stored). Stance auto-populates from the user's follow mode (`pocket_boost` → YES, `pocket_veto` → NO); clears if they unfollow. Recipient (house/senate) is derived from the bill's chamber. Calls `POST /{bill_id}/draft-letter` and renders the plain-text draft in a readonly textarea with a copy-to-clipboard button.
**`BillCard.tsx`** **`BillCard.tsx`**
Compact bill preview showing bill ID, title, sponsor with party badge, latest action date, and status. Compact bill preview showing bill ID, title, sponsor with party badge, latest action date, status, and a text availability indicator: `Brief` (green, analysis done) / `Pending` (amber, text retrieved but not yet analysed) / `No text` (muted, nothing published on Congress.gov).
**`TrendChart.tsx`** **`TrendChart.tsx`**
Line chart of `composite_score` over time with tooltip breakdown of each data source. Line chart of `composite_score` over time with tooltip breakdown of each data source.
@@ -805,7 +834,7 @@ Separate queues prevent a flood of LLM tasks from blocking time-sensitive pollin
All LLM providers implement the same interface. Switching providers is a single admin setting change — no code changes, no restart required (the factory reads from DB on each task invocation). All LLM providers implement the same interface. Switching providers is a single admin setting change — no code changes, no restart required (the factory reads from DB on each task invocation).
### JSONB for Flexible Brief Storage ### JSONB for Flexible Brief Storage
`key_points`, `risks`, `deadlines`, `topic_tags` are stored as JSONB. This means the schema change from `list[str]` to `list[{text, citation, quote}]` required no migration — only the LLM prompt and application code changed. Old string-format briefs and new cited-object briefs coexist in the same column. `key_points`, `risks`, `deadlines`, `topic_tags` are stored as JSONB. This means schema changes (adding `citation`/`quote` in v0.2.0, adding `label` in v0.6.0) required no migrations — only the LLM prompt and application code changed. Old string-format briefs, cited-object briefs without labels, and fully-labelled briefs all coexist in the same column and render correctly at each fidelity level.
### Redis-backed Beat Schedule (RedBeat) ### Redis-backed Beat Schedule (RedBeat)
The Celery Beat schedule is stored in Redis rather than in memory. This means the beat scheduler can restart without losing schedule state or double-firing tasks. The Celery Beat schedule is stored in Redis rather than in memory. This means the beat scheduler can restart without losing schedule state or double-firing tasks.
@@ -915,6 +944,55 @@ Nginx uses `resolver 127.0.0.11 valid=10s` (Docker's internal DNS) so upstream c
- `introduced_date` shown conditionally (not rendered when null, preventing "Introduced: —") - `introduced_date` shown conditionally (not rendered when null, preventing "Introduced: —")
- Admin reprocess endpoint: `POST /api/admin/bills/{bill_id}/reprocess` - Admin reprocess endpoint: `POST /api/admin/bills/{bill_id}/reprocess`
### v0.5.0 — Follow Modes, Public Browsing & Draft Letter Generator
**Follow Modes:**
- `follow_mode` column on `follows` table: `neutral | pocket_veto | pocket_boost`
- `FollowButton` replaced with a mode-selector dropdown (shield/zap/heart icons, descriptions for each mode)
- `pocket_veto` — alert only on advancement milestones; `pocket_boost` — all changes + action prompts
- Mode stored per-follow; respected by notification dispatcher
**Public Browsing:**
- Unauthenticated guests can browse bills, members, topics, and the trending dashboard
- `AuthModal` gates follow and other interactive actions
- Sidebar and nav adapt to guest state (no email/logout shown)
- All public endpoints already auth-free; guard refactored to allow guest reads
**Draft Constituent Letter Generator (email_gen):**
- `DraftLetterPanel.tsx` — collapsible UI below `BriefPanel` for bills with a brief
- User selects up to 3 cited points from the brief, picks stance (YES/NO), tone, optional ZIP (not stored)
- Stance pre-fills from follow mode; clears on unfollow (ref-tracked, not effect-guarded)
- Recipient derived from bill chamber — no dropdown needed
- `POST /api/bills/{bill_id}/draft-letter` endpoint: reads LLM provider/model from `AppSetting` (respects Settings page), wraps LLM errors with human-readable messages (quota, rate limit, auth)
- `generate_text(prompt) → str` added to `LLMProvider` ABC and all four providers
**Bill Text Status Indicators:**
- `has_document` field added to `BillSchema` (list endpoint) via a single batch `SELECT DISTINCT` — no per-card queries
- `BillCard` shows: `Brief` (green) / `Pending` (amber) / `No text` (muted) based on brief + document state
### v0.6.0 — Phase 2: Change-driven Alerts & Fact/Inference Labeling
**Change-driven Alerts:**
- `notification_utils.py` milestone keyword list expanded: added `"markup"` (markup sessions) and `"conference"` (conference committee)
- New `is_referral_action()` classifier for committee referrals (`"referred to"`)
- Two-tier notification system: `milestone_tier` field in `NotificationEvent.payload`
- `"progress"` — high-signal milestones (passed, signed, markup, etc.): all follow types notified
- `"referral"` — committee referral: pocket_veto and pocket_boost notified; neutral silently dropped
- **Topic followers now receive `bill_updated` milestone notifications** — previously they only received `new_document`/`new_amendment` events. Fixed by querying the bill's latest brief for `topic_tags` inside `_update_bill_if_changed()`
- All three follow types (bill, sponsor, topic) covered for both tiers
**Fact vs Inference Labeling:**
- `label: "cited_fact" | "inference"` added to every cited key_point and risk in the LLM JSON schema
- System prompt updated for all four providers (OpenAI, Anthropic, Gemini, Ollama)
- UI: neutral "Inferred" badge shown next to inference items in `AIBriefCard`; cited_fact items render cleanly without a badge
- `backfill_brief_labels` Celery task: classifies existing cited points in-place — one compact LLM call per brief (all points batched), updates JSONB with `flag_modified`, no brief re-generation
- `POST /api/admin/backfill-labels` endpoint + "Backfill Fact/Inference Labels" button in Admin panel
- `unlabeled_briefs` counter added to `/api/admin/stats` and pipeline breakdown table
**Admin Panel Cleanup:**
- Manual Controls split into two sections: always-visible recurring controls (Poll, Members, Trends, Actions, Resume) and a collapsible **Maintenance** section for one-time backfill tasks
- Maintenance section header shows "⚠ action needed" when any backfill has a non-zero count
### v0.2.2 — Sponsor Linking & Search Fixes ### v0.2.2 — Sponsor Linking & Search Fixes
- **Root cause fixed:** Congress.gov list API does not return sponsor data — only the detail endpoint does. Poller now calls the detail endpoint for each new bill to get the sponsor and populate `bill.sponsor_id` - **Root cause fixed:** Congress.gov list API does not return sponsor data — only the detail endpoint does. Poller now calls the detail endpoint for each new bill to get the sponsor and populate `bill.sponsor_id`
- **Backfill task:** `backfill_sponsor_ids` Celery task + `/api/admin/backfill-sponsors` endpoint + "Backfill Sponsors" button in Admin UI — fixes existing bills with `NULL` sponsor_id (~10 req/sec, ~3 min for 1,600 bills) - **Backfill task:** `backfill_sponsor_ids` Celery task + `/api/admin/backfill-sponsors` endpoint + "Backfill Sponsors" button in Admin UI — fixes existing bills with `NULL` sponsor_id (~10 req/sec, ~3 min for 1,600 bills)

View File

@@ -31,6 +31,10 @@
- [x] Backfill All Actions — admin task to fetch action history for all pre-existing bills - [x] Backfill All Actions — admin task to fetch action history for all pre-existing bills
- [x] Notifications (Phase 1) — ntfy dispatch, RSS feed, per-user settings UI, 5-min dispatcher beat task - [x] Notifications (Phase 1) — ntfy dispatch, RSS feed, per-user settings UI, 5-min dispatcher beat task
- [x] Brief Regeneration UI — admin button to delete existing briefs for a bill and re-queue LLM processing. Useful for improving citation/diff logic without a full re-poll. (Backend reprocess endpoint already exists.) - [x] Brief Regeneration UI — admin button to delete existing briefs for a bill and re-queue LLM processing. Useful for improving citation/diff logic without a full re-poll. (Backend reprocess endpoint already exists.)
- [x] Follow Modes — `neutral | pocket_veto | pocket_boost` on the `follows` table; FollowButton mode selector with descriptions and tooltips
- [x] Public Browsing — unauthenticated guests browse bills, members, topics, and trending dashboard; AuthModal gates follow/interactive actions; sidebar and nav adapt to guest state
- [x] Draft Letter Generator — collapsible panel on bill detail pages; select up to 3 cited brief points, stance auto-fills from follow mode, recipient derived from chamber, ZIP optional and never stored; calls configured LLM provider
- [x] Bill Text Status Indicators — BillCard shows Brief / Pending / No text badge; backed by a single batch query on the list endpoint
--- ---
@@ -40,8 +44,8 @@
### Phase 2 — High Impact *(can run in parallel after Phase 1)* ### Phase 2 — High Impact *(can run in parallel after Phase 1)*
- [ ] **Change-driven Alerts**emit `notification_event` from poller/document fetcher on material changes: new doc version, substitute text, committee report, vote scheduled/result. Filter out procedural-only action text. Fan out to ntfy + RSS. - [x] **Change-driven Alerts** — milestone keywords expanded (markup, conference, referral tier); topic followers now receive bill_updated milestone events; committee referral events delivered to pocket_veto/boost but suppressed for neutral; all three follow types covered for both tiers.
- [ ] **Fact vs Inference Labeling** add `label: "cited_fact" | "inference"` + optional `confidence` field to each `key_point` and `risk` in the LLM JSON schema. Prompt engineering change + BillBrief schema migration. UI: small badge on each bullet (no color politics — neutral labels only). - [x] **Fact vs Inference Labeling**`label: "cited_fact" | "inference"` on every cited key_point and risk; prompt engineering updated for all providers; "Inferred" badge in citation UI; backfill task for existing briefs.
--- ---
@@ -76,10 +80,9 @@
- [ ] **Source Viewer Option B** — in-app bill text viewer with cited passage highlighted and scroll-to-anchor. Deferred pending UX review of Option A (GovInfo link). - [ ] **Source Viewer Option B** — in-app bill text viewer with cited passage highlighted and scroll-to-anchor. Deferred pending UX review of Option A (GovInfo link).
- [ ] **Raw Diff Panel** — Python `difflib` diff between stored document versions, shown as collapsible "Raw Changes" below amendment brief. Zero API calls. Deferred — AI amendment brief is the primary "what changed" story. - [ ] **Raw Diff Panel** — Python `difflib` diff between stored document versions, shown as collapsible "Raw Changes" below amendment brief. Zero API calls. Deferred — AI amendment brief is the primary "what changed" story.
- [ ] **Shareable Collection Subscriptions** — "Follow this collection" mechanic so other users can subscribe to a public collection and get its bills added to their feed. - [ ] **Shareable Collection Subscriptions** — "Follow this collection" mechanic so other users can subscribe to a public collection and get its bills added to their feed.
- [ ] Pocket Veto mode (follow stance) — toggle on a bill to treat it as “I dont want this to pass”; adds to watchlist and triggers milestone alerts (committee report-out, calendared, vote scheduled, passed chamber, etc.) - [x] Pocket Veto mode (follow stance) — toggle on a bill to treat it as “I dont want this to pass”; adds to watchlist and triggers milestone alerts
- [ ] Pocket Veto notification rules — alert only on advancement milestones + failure outcomes (failed committee / failed floor / stalled) - [x] Follow modes — Neutral + Pocket Veto + Pocket Boost; FollowButton is a mode selector with explanation tooltips
- [ ] Follow modes — support Neutral (normal follow) + Pocket Veto now; optional Pocket Boost later - [ ] Pocket Veto notification rules — alert only on advancement milestones + failure outcomes (failed committee / failed floor / stalled) — notification dispatcher needs to filter by follow_mode
- [ ] UI: FollowButton becomes FollowMode selector (Neutral / Pocket Veto) with explanation tooltip
### PocketVeto function ### PocketVeto function
@@ -162,3 +165,51 @@ Then alert rules can be:
- pocket_boost: “action points” + milestones - pocket_boost: “action points” + milestones
Yes — thats a solid idea **if its done as a “welcome + how it works” nudge**, not an annoying pop-up that blocks the UI.
A toast can work, but for a first-time user youll usually get better results with a **dismissible banner** or a **one-time “welcome” card** on the dashboard, because:
* toasts disappear (people miss them)
* first-run onboarding usually needs at least one click (“Got it” / “Start here”)
### Best pattern (low effort, high impact)
**First visit → show a dismissible Welcome card/banner** (top of dashboard) with:
* 1 sentence purpose
* 3 bullets of key features
* 2 buttons: **“Add my first follow”** and **“See a demo bill”**
* “Dont show again” checkbox (or implicit on dismiss)
You can still use a toast, but make it:
* sticky until dismissed
* or paired with a banner/card
### What it should say (copy you can paste)
**Title:** Welcome to PocketVeto
**Body (tight):**
* Follow bills, members, or topics (low-noise)
* See *what changed* in plain English
* Verify every claim with **Back to Source** citations
**Buttons:** “Add a follow” | “Load demo”
### Implementation detail (no creepy tracking)
Store a simple flag:
* `localStorage.setItem("pv_seen_welcome", "1")`
Dont store it server-side unless you already have user accounts and its part of preferences.
### Backlog item (checkboxes)
* [ ] First-visit welcome UI (banner/card + optional toast)
* [ ] Dismiss + “dont show again” (localStorage)
* [ ] CTA: Add first follow
* [ ] CTA: Load demo data (optional)
* [ ] Link: “How it works” page/modal (optional)
If you tell me your UI stack (sounds like Next.js + shadcn/ui), I can give you a drop-in component for a clean welcome card + toast behavior.

View File

@@ -134,6 +134,23 @@ async def get_stats(
bills_missing_actions = (await db.execute( bills_missing_actions = (await db.execute(
text("SELECT COUNT(*) FROM bills WHERE actions_fetched_at IS NULL") text("SELECT COUNT(*) FROM bills WHERE actions_fetched_at IS NULL")
)).scalar() )).scalar()
# Cited brief points (objects) that have no label yet
unlabeled_briefs = (await db.execute(
text("""
SELECT COUNT(*) FROM bill_briefs
WHERE (
key_points IS NOT NULL AND EXISTS (
SELECT 1 FROM jsonb_array_elements(key_points) AS p
WHERE jsonb_typeof(p) = 'object' AND (p->>'label') IS NULL
)
) OR (
risks IS NOT NULL AND EXISTS (
SELECT 1 FROM jsonb_array_elements(risks) AS r
WHERE jsonb_typeof(r) = 'object' AND (r->>'label') IS NULL
)
)
""")
)).scalar()
return { return {
"total_bills": total_bills, "total_bills": total_bills,
"docs_fetched": docs_fetched, "docs_fetched": docs_fetched,
@@ -146,6 +163,7 @@ async def get_stats(
"bills_missing_sponsor": bills_missing_sponsor, "bills_missing_sponsor": bills_missing_sponsor,
"bills_missing_metadata": bills_missing_metadata, "bills_missing_metadata": bills_missing_metadata,
"bills_missing_actions": bills_missing_actions, "bills_missing_actions": bills_missing_actions,
"unlabeled_briefs": unlabeled_briefs,
"remaining": total_bills - total_briefs, "remaining": total_bills - total_briefs,
} }
@@ -204,6 +222,14 @@ async def backfill_metadata(current_user: User = Depends(get_current_admin)):
return {"task_id": task.id, "status": "queued"} return {"task_id": task.id, "status": "queued"}
@router.post("/backfill-labels")
async def backfill_labels(current_user: User = Depends(get_current_admin)):
"""Classify existing cited brief points as fact or inference without re-generating briefs."""
from app.workers.llm_processor import backfill_brief_labels
task = backfill_brief_labels.delay()
return {"task_id": task.id, "status": "queued"}
@router.post("/resume-analysis") @router.post("/resume-analysis")
async def resume_analysis(current_user: User = Depends(get_current_admin)): async def resume_analysis(current_user: User = Depends(get_current_admin)):
"""Re-queue LLM processing for docs with no brief, and document fetching for bills with no doc.""" """Re-queue LLM processing for docs with no brief, and document fetching for bills with no doc."""

View File

@@ -22,10 +22,10 @@ Always respond with valid JSON matching exactly this schema:
{ {
"summary": "2-4 paragraph plain-language summary of what this bill does", "summary": "2-4 paragraph plain-language summary of what this bill does",
"key_points": [ "key_points": [
{"text": "specific concrete fact", "citation": "Section X(y)", "quote": "verbatim excerpt from bill ≤80 words"} {"text": "specific concrete fact", "citation": "Section X(y)", "quote": "verbatim excerpt from bill ≤80 words", "label": "cited_fact"}
], ],
"risks": [ "risks": [
{"text": "legitimate concern or challenge", "citation": "Section X(y)", "quote": "verbatim excerpt from bill ≤80 words"} {"text": "legitimate concern or challenge", "citation": "Section X(y)", "quote": "verbatim excerpt from bill ≤80 words", "label": "cited_fact"}
], ],
"deadlines": [{"date": "YYYY-MM-DD or null", "description": "what happens on this date"}], "deadlines": [{"date": "YYYY-MM-DD or null", "description": "what happens on this date"}],
"topic_tags": ["healthcare", "taxation"] "topic_tags": ["healthcare", "taxation"]
@@ -35,10 +35,14 @@ Rules:
- summary: Explain WHAT the bill does, not whether it is good or bad. Be factual and complete. - summary: Explain WHAT the bill does, not whether it is good or bad. Be factual and complete.
- key_points: 5-10 specific, concrete things the bill changes, authorizes, or appropriates. \ - key_points: 5-10 specific, concrete things the bill changes, authorizes, or appropriates. \
Each item MUST include "text" (your claim), "citation" (the section number, e.g. "Section 301(a)(2)"), \ Each item MUST include "text" (your claim), "citation" (the section number, e.g. "Section 301(a)(2)"), \
and "quote" (a verbatim excerpt of ≤80 words from that section that supports your claim). "quote" (a verbatim excerpt of ≤80 words from that section that supports your claim), and "label".
- risks: Legitimate concerns from any perspective — costs, implementation challenges, \ - risks: Legitimate concerns from any perspective — costs, implementation challenges, \
constitutional questions, unintended consequences. Include at least 2 even for benign bills. \ constitutional questions, unintended consequences. Include at least 2 even for benign bills. \
Each item MUST include "text", "citation", and "quote" just like key_points. Each item MUST include "text", "citation", "quote", and "label" just like key_points.
- label: "cited_fact" if the claim is directly and explicitly stated in the quoted text. \
"inference" if the claim is an analytical interpretation, projection, or implication that goes \
beyond what the text literally says (e.g. projected costs, likely downstream effects, \
constitutional questions). When in doubt, use "inference".
- deadlines: Only include if explicitly stated in the text. Use null for date if a deadline \ - deadlines: Only include if explicitly stated in the text. Use null for date if a deadline \
is mentioned without a specific date. Empty list if none. is mentioned without a specific date. Empty list if none.
- topic_tags: 3-8 lowercase tags. Prefer these standard tags: healthcare, taxation, defense, \ - topic_tags: 3-8 lowercase tags. Prefer these standard tags: healthcare, taxation, defense, \
@@ -90,10 +94,10 @@ Always respond with valid JSON matching exactly this schema:
{ {
"summary": "2-3 paragraph plain-language description of what changed in this version", "summary": "2-3 paragraph plain-language description of what changed in this version",
"key_points": [ "key_points": [
{"text": "specific change", "citation": "Section X(y)", "quote": "verbatim excerpt from new version ≤80 words"} {"text": "specific change", "citation": "Section X(y)", "quote": "verbatim excerpt from new version ≤80 words", "label": "cited_fact"}
], ],
"risks": [ "risks": [
{"text": "new concern introduced by this change", "citation": "Section X(y)", "quote": "verbatim excerpt from new version ≤80 words"} {"text": "new concern introduced by this change", "citation": "Section X(y)", "quote": "verbatim excerpt from new version ≤80 words", "label": "cited_fact"}
], ],
"deadlines": [{"date": "YYYY-MM-DD or null", "description": "new deadline added"}], "deadlines": [{"date": "YYYY-MM-DD or null", "description": "new deadline added"}],
"topic_tags": ["healthcare", "taxation"] "topic_tags": ["healthcare", "taxation"]
@@ -103,9 +107,12 @@ Rules:
- summary: Focus ONLY on what is different from the previous version. Be specific. - summary: Focus ONLY on what is different from the previous version. Be specific.
- key_points: List concrete additions, removals, or modifications in this version. \ - key_points: List concrete additions, removals, or modifications in this version. \
Each item MUST include "text" (your claim), "citation" (the section number, e.g. "Section 301(a)(2)"), \ Each item MUST include "text" (your claim), "citation" (the section number, e.g. "Section 301(a)(2)"), \
and "quote" (a verbatim excerpt of ≤80 words from the NEW version that supports your claim). "quote" (a verbatim excerpt of ≤80 words from the NEW version that supports your claim), and "label".
- risks: Only include risks that are new or changed relative to the previous version. \ - risks: Only include risks that are new or changed relative to the previous version. \
Each item MUST include "text", "citation", and "quote" just like key_points. Each item MUST include "text", "citation", "quote", and "label" just like key_points.
- label: "cited_fact" if the claim is directly and explicitly stated in the quoted text. \
"inference" if the claim is an analytical interpretation, projection, or implication that goes \
beyond what the text literally says. When in doubt, use "inference".
- deadlines: Only new or changed deadlines. Empty list if none. - deadlines: Only new or changed deadlines. Empty list if none.
- topic_tags: Same standard tags as before — include any new topics this version adds. - topic_tags: Same standard tags as before — include any new topics this version adds.

View File

@@ -338,12 +338,29 @@ def _update_bill_if_changed(db, existing: Bill, parsed: dict) -> bool:
from app.workers.notification_utils import ( from app.workers.notification_utils import (
emit_bill_notification, emit_bill_notification,
emit_member_follow_notifications, emit_member_follow_notifications,
emit_topic_follow_notifications,
is_milestone_action, is_milestone_action,
is_referral_action,
)
action_text = parsed.get("latest_action_text", "")
is_milestone = is_milestone_action(action_text)
is_referral = not is_milestone and is_referral_action(action_text)
if is_milestone or is_referral:
tier = "progress" if is_milestone else "referral"
emit_bill_notification(db, existing, "bill_updated", action_text, milestone_tier=tier)
emit_member_follow_notifications(db, existing, "bill_updated", action_text, milestone_tier=tier)
# Topic followers — pull tags from the bill's latest brief
from app.models.brief import BillBrief
latest_brief = (
db.query(BillBrief)
.filter_by(bill_id=existing.bill_id)
.order_by(BillBrief.created_at.desc())
.first()
)
topic_tags = latest_brief.topic_tags or [] if latest_brief else []
emit_topic_follow_notifications(
db, existing, "bill_updated", action_text, topic_tags, milestone_tier=tier
) )
if is_milestone_action(parsed.get("latest_action_text", "")):
action_text = parsed["latest_action_text"]
emit_bill_notification(db, existing, "bill_updated", action_text)
emit_member_follow_notifications(db, existing, "bill_updated", action_text)
return changed return changed

View File

@@ -181,6 +181,113 @@ def backfill_brief_citations(self):
db.close() db.close()
@celery_app.task(bind=True, name="app.workers.llm_processor.backfill_brief_labels")
def backfill_brief_labels(self):
"""
Add fact/inference labels to existing cited brief points without re-generating them.
Sends one compact classification call per brief (all unlabeled points batched).
Skips briefs already fully labeled and plain-string points (no quote to classify).
"""
import json
from sqlalchemy.orm.attributes import flag_modified
from app.models.setting import AppSetting
db = get_sync_db()
try:
unlabeled_ids = db.execute(text("""
SELECT id FROM bill_briefs
WHERE (
key_points IS NOT NULL AND EXISTS (
SELECT 1 FROM jsonb_array_elements(key_points) AS p
WHERE jsonb_typeof(p) = 'object' AND (p->>'label') IS NULL
)
) OR (
risks IS NOT NULL AND EXISTS (
SELECT 1 FROM jsonb_array_elements(risks) AS r
WHERE jsonb_typeof(r) = 'object' AND (r->>'label') IS NULL
)
)
""")).fetchall()
total = len(unlabeled_ids)
updated = 0
skipped = 0
prov_row = db.get(AppSetting, "llm_provider")
model_row = db.get(AppSetting, "llm_model")
provider = get_llm_provider(
prov_row.value if prov_row else None,
model_row.value if model_row else None,
)
for row in unlabeled_ids:
brief = db.get(BillBrief, row.id)
if not brief:
skipped += 1
continue
# Collect all unlabeled cited points across both fields
to_classify: list[tuple[str, int, dict]] = []
for field_name in ("key_points", "risks"):
for i, p in enumerate(getattr(brief, field_name) or []):
if isinstance(p, dict) and p.get("label") is None:
to_classify.append((field_name, i, p))
if not to_classify:
skipped += 1
continue
lines = [
f'{i + 1}. TEXT: "{p["text"]}" | QUOTE: "{p.get("quote", "")}"'
for i, (_, __, p) in enumerate(to_classify)
]
prompt = (
"Classify each item as 'cited_fact' or 'inference'.\n"
"cited_fact = the claim is explicitly and directly stated in the quoted text.\n"
"inference = analytical interpretation, projection, or implication not literally stated.\n\n"
"Return ONLY a JSON array of strings, one per item, in order. No explanation.\n\n"
"Items:\n" + "\n".join(lines)
)
try:
raw = provider.generate_text(prompt).strip()
if raw.startswith("```"):
raw = raw.split("```")[1]
if raw.startswith("json"):
raw = raw[4:]
labels = json.loads(raw.strip())
if not isinstance(labels, list) or len(labels) != len(to_classify):
logger.warning(f"Brief {brief.id}: label count mismatch, skipping")
skipped += 1
continue
except Exception as exc:
logger.warning(f"Brief {brief.id}: classification failed: {exc}")
skipped += 1
time.sleep(0.5)
continue
fields_modified: set[str] = set()
for (field_name, point_idx, _), label in zip(to_classify, labels):
if label in ("cited_fact", "inference"):
getattr(brief, field_name)[point_idx]["label"] = label
fields_modified.add(field_name)
for field_name in fields_modified:
flag_modified(brief, field_name)
db.commit()
updated += 1
time.sleep(0.2)
logger.info(
f"backfill_brief_labels: {total} briefs found, "
f"{updated} updated, {skipped} skipped"
)
return {"total": total, "updated": updated, "skipped": skipped}
finally:
db.close()
@celery_app.task(bind=True, name="app.workers.llm_processor.resume_pending_analysis") @celery_app.task(bind=True, name="app.workers.llm_processor.resume_pending_analysis")
def resume_pending_analysis(self): def resume_pending_analysis(self):
""" """

View File

@@ -94,6 +94,13 @@ def dispatch_notifications(self):
db.commit() db.commit()
continue continue
# Referral-tier events (committee referrals) are noisy for neutral follows;
# pocket_veto and pocket_boost users want them as early warnings
if follow_mode == "neutral" and (event.payload or {}).get("milestone_tier") == "referral":
event.dispatched_at = now
db.commit()
continue
prefs = user.notification_prefs or {} prefs = user.notification_prefs or {}
ntfy_url = prefs.get("ntfy_topic_url", "").strip() ntfy_url = prefs.get("ntfy_topic_url", "").strip()
ntfy_auth_method = prefs.get("ntfy_auth_method", "none") ntfy_auth_method = prefs.get("ntfy_auth_method", "none")

View File

@@ -12,6 +12,13 @@ _MILESTONE_KEYWORDS = [
"reported by", "discharged", "reported by", "discharged",
"placed on", # placed on calendar "placed on", # placed on calendar
"cloture", "roll call", "cloture", "roll call",
"markup", # markup session scheduled/completed
"conference", # conference committee activity
]
# Committee referral — meaningful for pocket_veto/boost but noisy for neutral
_REFERRAL_KEYWORDS = [
"referred to",
] ]
# Events created within this window for the same (user, bill, event_type) are suppressed # Events created within this window for the same (user, bill, event_type) are suppressed
@@ -23,7 +30,12 @@ def is_milestone_action(action_text: str) -> bool:
return any(kw in t for kw in _MILESTONE_KEYWORDS) return any(kw in t for kw in _MILESTONE_KEYWORDS)
def _build_payload(bill, action_summary: str) -> dict: def is_referral_action(action_text: str) -> bool:
t = (action_text or "").lower()
return any(kw in t for kw in _REFERRAL_KEYWORDS)
def _build_payload(bill, action_summary: str, milestone_tier: str = "progress") -> dict:
from app.config import settings from app.config import settings
base_url = (settings.PUBLIC_URL or settings.LOCAL_URL).rstrip("/") base_url = (settings.PUBLIC_URL or settings.LOCAL_URL).rstrip("/")
return { return {
@@ -31,6 +43,7 @@ def _build_payload(bill, action_summary: str) -> dict:
"bill_label": f"{bill.bill_type.upper()} {bill.bill_number}", "bill_label": f"{bill.bill_type.upper()} {bill.bill_number}",
"brief_summary": (action_summary or "")[:300], "brief_summary": (action_summary or "")[:300],
"bill_url": f"{base_url}/bills/{bill.bill_id}", "bill_url": f"{base_url}/bills/{bill.bill_id}",
"milestone_tier": milestone_tier,
} }
@@ -45,7 +58,9 @@ def _is_duplicate(db, user_id: int, bill_id: str, event_type: str) -> bool:
).filter(NotificationEvent.created_at > cutoff).first() is not None ).filter(NotificationEvent.created_at > cutoff).first() is not None
def emit_bill_notification(db, bill, event_type: str, action_summary: str) -> int: def emit_bill_notification(
db, bill, event_type: str, action_summary: str, milestone_tier: str = "progress"
) -> int:
"""Create NotificationEvent rows for every user following this bill. Returns count.""" """Create NotificationEvent rows for every user following this bill. Returns count."""
from app.models.follow import Follow from app.models.follow import Follow
from app.models.notification import NotificationEvent from app.models.notification import NotificationEvent
@@ -54,7 +69,7 @@ def emit_bill_notification(db, bill, event_type: str, action_summary: str) -> in
if not followers: if not followers:
return 0 return 0
payload = _build_payload(bill, action_summary) payload = _build_payload(bill, action_summary, milestone_tier)
count = 0 count = 0
for follow in followers: for follow in followers:
if _is_duplicate(db, follow.user_id, bill.bill_id, event_type): if _is_duplicate(db, follow.user_id, bill.bill_id, event_type):
@@ -71,7 +86,9 @@ def emit_bill_notification(db, bill, event_type: str, action_summary: str) -> in
return count return count
def emit_member_follow_notifications(db, bill, event_type: str, action_summary: str) -> int: def emit_member_follow_notifications(
db, bill, event_type: str, action_summary: str, milestone_tier: str = "progress"
) -> int:
"""Notify users following the bill's sponsor (dedup prevents double-alerts for bill+member followers).""" """Notify users following the bill's sponsor (dedup prevents double-alerts for bill+member followers)."""
if not bill.sponsor_id: if not bill.sponsor_id:
return 0 return 0
@@ -83,7 +100,7 @@ def emit_member_follow_notifications(db, bill, event_type: str, action_summary:
if not followers: if not followers:
return 0 return 0
payload = _build_payload(bill, action_summary) payload = _build_payload(bill, action_summary, milestone_tier)
count = 0 count = 0
for follow in followers: for follow in followers:
if _is_duplicate(db, follow.user_id, bill.bill_id, event_type): if _is_duplicate(db, follow.user_id, bill.bill_id, event_type):
@@ -100,7 +117,10 @@ def emit_member_follow_notifications(db, bill, event_type: str, action_summary:
return count return count
def emit_topic_follow_notifications(db, bill, event_type: str, action_summary: str, topic_tags: list) -> int: def emit_topic_follow_notifications(
db, bill, event_type: str, action_summary: str, topic_tags: list,
milestone_tier: str = "progress",
) -> int:
"""Notify users following any of the bill's topic tags.""" """Notify users following any of the bill's topic tags."""
if not topic_tags: if not topic_tags:
return 0 return 0
@@ -120,7 +140,7 @@ def emit_topic_follow_notifications(db, bill, event_type: str, action_summary: s
if not followers: if not followers:
return 0 return 0
payload = _build_payload(bill, action_summary) payload = _build_payload(bill, action_summary, milestone_tier)
count = 0 count = 0
for follow in followers: for follow in followers:
if _is_duplicate(db, follow.user_id, bill.bill_id, event_type): if _is_duplicate(db, follow.user_id, bill.bill_id, event_type):

View File

@@ -17,6 +17,9 @@ import {
Bell, Bell,
Shield, Shield,
Zap, Zap,
ChevronDown,
ChevronRight,
Wrench,
} from "lucide-react"; } from "lucide-react";
import Link from "next/link"; import Link from "next/link";
import { settingsAPI, adminAPI, notificationsAPI, type AdminUser, type LLMModel, type ApiHealthResult } from "@/lib/api"; import { settingsAPI, adminAPI, notificationsAPI, type AdminUser, type LLMModel, type ApiHealthResult } from "@/lib/api";
@@ -134,6 +137,7 @@ export default function SettingsPage() {
const [taskIds, setTaskIds] = useState<Record<string, string>>({}); const [taskIds, setTaskIds] = useState<Record<string, string>>({});
const [taskStatuses, setTaskStatuses] = useState<Record<string, "running" | "done" | "error">>({}); const [taskStatuses, setTaskStatuses] = useState<Record<string, "running" | "done" | "error">>({});
const [confirmDelete, setConfirmDelete] = useState<number | null>(null); const [confirmDelete, setConfirmDelete] = useState<number | null>(null);
const [showMaintenance, setShowMaintenance] = useState(false);
const testLLM = async () => { const testLLM = async () => {
setTesting(true); setTesting(true);
@@ -316,6 +320,7 @@ export default function SettingsPage() {
{ label: "AI briefs generated", value: stats.briefs_generated, color: "text-green-600 dark:text-green-400", icon: "✅" }, { label: "AI briefs generated", value: stats.briefs_generated, color: "text-green-600 dark:text-green-400", icon: "✅" },
{ label: "Pending LLM analysis", value: stats.pending_llm, color: stats.pending_llm > 0 ? "text-amber-600 dark:text-amber-400" : "text-muted-foreground", icon: "🔄", action: stats.pending_llm > 0 ? "Resume Analysis" : undefined }, { label: "Pending LLM analysis", value: stats.pending_llm, color: stats.pending_llm > 0 ? "text-amber-600 dark:text-amber-400" : "text-muted-foreground", icon: "🔄", action: stats.pending_llm > 0 ? "Resume Analysis" : undefined },
{ label: "Briefs missing citations", value: stats.uncited_briefs, color: stats.uncited_briefs > 0 ? "text-amber-600 dark:text-amber-400" : "text-muted-foreground", icon: "⚠️", action: stats.uncited_briefs > 0 ? "Backfill Citations" : undefined }, { label: "Briefs missing citations", value: stats.uncited_briefs, color: stats.uncited_briefs > 0 ? "text-amber-600 dark:text-amber-400" : "text-muted-foreground", icon: "⚠️", action: stats.uncited_briefs > 0 ? "Backfill Citations" : undefined },
{ label: "Briefs with unlabeled points", value: stats.unlabeled_briefs, color: stats.unlabeled_briefs > 0 ? "text-amber-600 dark:text-amber-400" : "text-muted-foreground", icon: "🏷️", action: stats.unlabeled_briefs > 0 ? "Backfill Labels" : undefined },
].map(({ label, value, color, icon, note, action }) => ( ].map(({ label, value, color, icon, note, action }) => (
<div key={label} className="flex items-center justify-between py-2.5 gap-3"> <div key={label} className="flex items-center justify-between py-2.5 gap-3">
<div className="flex items-center gap-2 min-w-0"> <div className="flex items-center gap-2 min-w-0">
@@ -637,82 +642,9 @@ export default function SettingsPage() {
{/* Manual Controls */} {/* Manual Controls */}
<section className="bg-card border border-border rounded-lg p-6 space-y-4"> <section className="bg-card border border-border rounded-lg p-6 space-y-4">
<h2 className="font-semibold">Manual Controls</h2> <h2 className="font-semibold">Manual Controls</h2>
<div className="divide-y divide-border">
{([ {(() => {
{ type ControlItem = {
key: "poll",
name: "Trigger Poll",
description: "Check Congress.gov for newly introduced or updated bills. Runs automatically on a schedule — use this to force an immediate sync.",
fn: adminAPI.triggerPoll,
status: "on-demand",
},
{
key: "members",
name: "Sync Members",
description: "Refresh all member profiles from Congress.gov including biography, current term, leadership roles, and contact information.",
fn: adminAPI.triggerMemberSync,
status: "on-demand",
},
{
key: "trends",
name: "Calculate Trends",
description: "Score bill and member newsworthiness by counting recent news headlines and Google search interest. Updates the trend charts.",
fn: adminAPI.triggerTrendScores,
status: "on-demand",
},
{
key: "actions",
name: "Fetch Bill Actions",
description: "Download the full legislative history (votes, referrals, amendments) for recently active bills and populate the timeline view.",
fn: adminAPI.triggerFetchActions,
status: "on-demand",
},
{
key: "backfill-actions",
name: "Backfill All Action Histories",
description: "One-time catch-up: fetch action histories for all bills that were imported before this feature existed. Run once to populate timelines across your full bill archive.",
fn: adminAPI.backfillAllActions,
status: stats ? (stats.bills_missing_actions > 0 ? "needed" : "ok") : "on-demand",
count: stats?.bills_missing_actions,
countLabel: "bills missing action history",
},
{
key: "sponsors",
name: "Backfill Sponsors",
description: "Link bill sponsors that weren't captured during the initial import. Safe to re-run — skips bills that already have a sponsor.",
fn: adminAPI.backfillSponsors,
status: stats ? (stats.bills_missing_sponsor > 0 ? "needed" : "ok") : "on-demand",
count: stats?.bills_missing_sponsor,
countLabel: "bills missing sponsor",
},
{
key: "metadata",
name: "Backfill Dates & Links",
description: "Fill in missing introduced dates, chamber assignments, and congress.gov links by re-fetching bill detail from Congress.gov.",
fn: adminAPI.backfillMetadata,
status: stats ? (stats.bills_missing_metadata > 0 ? "needed" : "ok") : "on-demand",
count: stats?.bills_missing_metadata,
countLabel: "bills missing metadata",
},
{
key: "citations",
name: "Backfill Citations",
description: "Regenerate AI briefs that were created before inline source citations were added. Deletes the old brief and re-runs LLM analysis using the already-stored bill text — no new Congress.gov calls.",
fn: adminAPI.backfillCitations,
status: stats ? (stats.uncited_briefs > 0 ? "needed" : "ok") : "on-demand",
count: stats?.uncited_briefs,
countLabel: "briefs need regeneration",
},
{
key: "resume",
name: "Resume Analysis",
description: "Restart AI brief generation for bills where processing stalled or failed (e.g. after an LLM quota outage). Also re-queues document fetching for bills that have no text yet.",
fn: adminAPI.resumeAnalysis,
status: stats ? (stats.pending_llm > 0 ? "needed" : "ok") : "on-demand",
count: stats?.pending_llm,
countLabel: "bills pending analysis",
},
] as Array<{
key: string; key: string;
name: string; name: string;
description: string; description: string;
@@ -720,7 +652,9 @@ export default function SettingsPage() {
status: "ok" | "needed" | "on-demand"; status: "ok" | "needed" | "on-demand";
count?: number; count?: number;
countLabel?: string; countLabel?: string;
}>).map(({ key, name, description, fn, status, count, countLabel }) => ( };
const renderRow = ({ key, name, description, fn, status, count, countLabel }: ControlItem) => (
<div key={key} className="flex items-start gap-3 py-3.5"> <div key={key} className="flex items-start gap-3 py-3.5">
<div className={`w-2.5 h-2.5 rounded-full mt-1 shrink-0 ${ <div className={`w-2.5 h-2.5 rounded-full mt-1 shrink-0 ${
status === "ok" ? "bg-green-500" status === "ok" ? "bg-green-500"
@@ -757,13 +691,133 @@ export default function SettingsPage() {
disabled={taskStatuses[key] === "running"} disabled={taskStatuses[key] === "running"}
className="shrink-0 flex items-center gap-1.5 px-3 py-1.5 text-xs bg-muted hover:bg-accent rounded-md transition-colors font-medium disabled:opacity-50 disabled:cursor-not-allowed" className="shrink-0 flex items-center gap-1.5 px-3 py-1.5 text-xs bg-muted hover:bg-accent rounded-md transition-colors font-medium disabled:opacity-50 disabled:cursor-not-allowed"
> >
{taskStatuses[key] === "running" ? ( {taskStatuses[key] === "running" ? <RefreshCw className="w-3 h-3 animate-spin" /> : "Run"}
<RefreshCw className="w-3 h-3 animate-spin" />
) : "Run"}
</button> </button>
</div> </div>
))} );
const recurring: ControlItem[] = [
{
key: "poll",
name: "Trigger Poll",
description: "Check Congress.gov for newly introduced or updated bills. Runs automatically on a schedule — use this to force an immediate sync.",
fn: adminAPI.triggerPoll,
status: "on-demand",
},
{
key: "members",
name: "Sync Members",
description: "Refresh all member profiles from Congress.gov including biography, current term, leadership roles, and contact information.",
fn: adminAPI.triggerMemberSync,
status: "on-demand",
},
{
key: "trends",
name: "Calculate Trends",
description: "Score bill and member newsworthiness by counting recent news headlines and Google search interest. Updates the trend charts.",
fn: adminAPI.triggerTrendScores,
status: "on-demand",
},
{
key: "actions",
name: "Fetch Bill Actions",
description: "Download the full legislative history (votes, referrals, amendments) for recently active bills and populate the timeline view.",
fn: adminAPI.triggerFetchActions,
status: "on-demand",
},
{
key: "resume",
name: "Resume Analysis",
description: "Restart AI brief generation for bills where processing stalled or failed (e.g. after an LLM quota outage). Also re-queues document fetching for bills that have no text yet.",
fn: adminAPI.resumeAnalysis,
status: stats ? (stats.pending_llm > 0 ? "needed" : "on-demand") : "on-demand",
count: stats?.pending_llm,
countLabel: "bills pending analysis",
},
];
const maintenance: ControlItem[] = [
{
key: "backfill-actions",
name: "Backfill All Action Histories",
description: "One-time catch-up: fetch action histories for all bills that were imported before this feature existed.",
fn: adminAPI.backfillAllActions,
status: stats ? (stats.bills_missing_actions > 0 ? "needed" : "ok") : "on-demand",
count: stats?.bills_missing_actions,
countLabel: "bills missing action history",
},
{
key: "sponsors",
name: "Backfill Sponsors",
description: "Link bill sponsors that weren't captured during the initial import. Safe to re-run — skips bills that already have a sponsor.",
fn: adminAPI.backfillSponsors,
status: stats ? (stats.bills_missing_sponsor > 0 ? "needed" : "ok") : "on-demand",
count: stats?.bills_missing_sponsor,
countLabel: "bills missing sponsor",
},
{
key: "metadata",
name: "Backfill Dates & Links",
description: "Fill in missing introduced dates, chamber assignments, and congress.gov links by re-fetching bill detail from Congress.gov.",
fn: adminAPI.backfillMetadata,
status: stats ? (stats.bills_missing_metadata > 0 ? "needed" : "ok") : "on-demand",
count: stats?.bills_missing_metadata,
countLabel: "bills missing metadata",
},
{
key: "citations",
name: "Backfill Citations",
description: "Regenerate AI briefs created before inline source citations were added. Deletes the old brief and re-runs LLM analysis using already-stored bill text.",
fn: adminAPI.backfillCitations,
status: stats ? (stats.uncited_briefs > 0 ? "needed" : "ok") : "on-demand",
count: stats?.uncited_briefs,
countLabel: "briefs need regeneration",
},
{
key: "labels",
name: "Backfill Fact/Inference Labels",
description: "Classify existing cited brief points as fact or inference. One compact LLM call per brief — no re-generation of summaries or citations.",
fn: adminAPI.backfillLabels,
status: stats ? (stats.unlabeled_briefs > 0 ? "needed" : "ok") : "on-demand",
count: stats?.unlabeled_briefs,
countLabel: "briefs with unlabeled points",
},
];
const maintenanceNeeded = maintenance.some((m) => m.status === "needed");
return (
<>
<div className="divide-y divide-border">
{recurring.map(renderRow)}
</div> </div>
{/* Maintenance subsection */}
<div className="border border-border rounded-md overflow-hidden">
<button
onClick={() => setShowMaintenance((v) => !v)}
className="w-full flex items-center justify-between px-4 py-3 text-sm font-medium bg-muted/50 hover:bg-muted transition-colors"
>
<span className="flex items-center gap-2">
<Wrench className="w-3.5 h-3.5 text-muted-foreground" />
Maintenance
{maintenanceNeeded && (
<span className="text-xs font-normal text-red-600 dark:text-red-400"> action needed</span>
)}
</span>
{showMaintenance
? <ChevronDown className="w-4 h-4 text-muted-foreground" />
: <ChevronRight className="w-4 h-4 text-muted-foreground" />}
</button>
{showMaintenance && (
<div className="divide-y divide-border px-4">
{maintenance.map(renderRow)}
</div>
)}
</div>
</>
);
})()}
</section> </section>
</div> </div>
); );

View File

@@ -31,6 +31,14 @@ function CitedItem({ point, icon, govinfo_url, openKey, activeKey, setActiveKey
<div className="flex items-start gap-2"> <div className="flex items-start gap-2">
<span className="mt-0.5 shrink-0">{icon}</span> <span className="mt-0.5 shrink-0">{icon}</span>
<span className="flex-1">{cited ? point.text : point}</span> <span className="flex-1">{cited ? point.text : point}</span>
{cited && point.label === "inference" && (
<span
title="This point is an analytical interpretation, not a literal statement from the bill text"
className="shrink-0 text-[10px] px-1.5 py-0.5 rounded border border-border text-muted-foreground font-sans leading-none"
>
Inferred
</span>
)}
{cited && ( {cited && (
<button <button
onClick={() => setActiveKey(isOpen ? null : openKey)} onClick={() => setActiveKey(isOpen ? null : openKey)}

View File

@@ -174,6 +174,7 @@ export interface AnalysisStats {
bills_missing_sponsor: number; bills_missing_sponsor: number;
bills_missing_metadata: number; bills_missing_metadata: number;
bills_missing_actions: number; bills_missing_actions: number;
unlabeled_briefs: number;
remaining: number; remaining: number;
} }
@@ -236,6 +237,8 @@ export const adminAPI = {
apiClient.post("/api/admin/backfill-all-actions").then((r) => r.data), apiClient.post("/api/admin/backfill-all-actions").then((r) => r.data),
backfillMetadata: () => backfillMetadata: () =>
apiClient.post("/api/admin/backfill-metadata").then((r) => r.data), apiClient.post("/api/admin/backfill-metadata").then((r) => r.data),
backfillLabels: () =>
apiClient.post("/api/admin/backfill-labels").then((r) => r.data),
resumeAnalysis: () => resumeAnalysis: () =>
apiClient.post("/api/admin/resume-analysis").then((r) => r.data), apiClient.post("/api/admin/resume-analysis").then((r) => r.data),
getApiHealth: () => getApiHealth: () =>

View File

@@ -58,6 +58,7 @@ export interface CitedPoint {
text: string; text: string;
citation: string; citation: string;
quote: string; quote: string;
label?: "cited_fact" | "inference";
} }
export interface BriefSchema { export interface BriefSchema {