feat: LLM Batch API — OpenAI + Anthropic 50% cost reduction (v0.9.8)
Submit up to 1000 unbriefed documents to the provider Batch API in one
shot instead of individual synchronous LLM calls. Results are polled
every 30 minutes via a new Celery beat task and imported automatically.
- New worker: llm_batch_processor.py
- submit_llm_batch: guards against duplicate batches, builds JSONL
(OpenAI) or request list (Anthropic), stores state in AppSetting
- poll_llm_batch_results: checks batch status, imports completed
results with idempotency, emits notifications + triggers news fetch
- celery_app: register worker, route to llm queue, beat every 30 min
- admin API: POST /submit-llm-batch + GET /llm-batch-status endpoints
- Frontend: submitLlmBatch + getLlmBatchStatus in adminAPI; settings
page shows batch control row (openai/anthropic only) with live
progress line while batch is processing
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -16,6 +16,7 @@ celery_app = Celery(
|
||||
"app.workers.trend_scorer",
|
||||
"app.workers.member_interest",
|
||||
"app.workers.notification_dispatcher",
|
||||
"app.workers.llm_batch_processor",
|
||||
],
|
||||
)
|
||||
|
||||
@@ -35,6 +36,7 @@ celery_app.conf.update(
|
||||
"app.workers.congress_poller.*": {"queue": "polling"},
|
||||
"app.workers.document_fetcher.*": {"queue": "documents"},
|
||||
"app.workers.llm_processor.*": {"queue": "llm"},
|
||||
"app.workers.llm_batch_processor.*": {"queue": "llm"},
|
||||
"app.workers.news_fetcher.*": {"queue": "news"},
|
||||
"app.workers.trend_scorer.*": {"queue": "news"},
|
||||
"app.workers.member_interest.*": {"queue": "news"},
|
||||
@@ -90,5 +92,9 @@ celery_app.conf.update(
|
||||
"task": "app.workers.notification_dispatcher.send_weekly_digest",
|
||||
"schedule": crontab(hour=8, minute=30, day_of_week=1), # Monday 8:30 AM UTC
|
||||
},
|
||||
"poll-llm-batch-results": {
|
||||
"task": "app.workers.llm_batch_processor.poll_llm_batch_results",
|
||||
"schedule": crontab(minute="*/30"),
|
||||
},
|
||||
},
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user