4c86a5b9ca
Self-hosted US Congress monitoring platform with AI policy briefs, bill/member/topic follows, ntfy + RSS + email notifications, alignment scoring, collections, and draft-letter generator. Authored by: Jack Levy
159 lines
3.8 KiB
Markdown
159 lines
3.8 KiB
Markdown
# Troubleshooting
|
|
|
|
Common issues encountered during development and deployment of PocketVeto.
|
|
|
|
---
|
|
|
|
## 502 Bad Gateway after rebuilding a container
|
|
|
|
**Symptom**
|
|
|
|
All API calls return 502. nginx error log shows:
|
|
|
|
```
|
|
connect() failed (111: Connection refused) while connecting to upstream,
|
|
upstream: "http://172.18.0.X:8000/api/..."
|
|
```
|
|
|
|
The IP in the error is the *old* IP of the container before the rebuild.
|
|
|
|
**Root cause**
|
|
|
|
When nginx uses `upstream` blocks, it resolves hostnames once at process startup and caches the result for the lifetime of the process. Rebuilding a container (e.g. `docker compose build api && docker compose up -d api`) assigns it a new Docker network IP. nginx still holds the old IP and all connections are refused.
|
|
|
|
**Immediate fix**
|
|
|
|
```bash
|
|
docker compose restart nginx
|
|
```
|
|
|
|
This forces nginx to re-resolve all upstream hostnames from Docker's internal DNS (`127.0.0.11`).
|
|
|
|
**Permanent fix (already applied)**
|
|
|
|
Replace `upstream` blocks with `set $variable` in `proxy_pass`. nginx only activates the `resolver` directive when a variable is used — making it re-resolve on each request cycle (every `valid=N` seconds).
|
|
|
|
```nginx
|
|
resolver 127.0.0.11 valid=10s ipv6=off;
|
|
|
|
# BAD — resolves once at startup, caches forever
|
|
upstream api {
|
|
server api:8000;
|
|
}
|
|
location /api/ {
|
|
proxy_pass http://api;
|
|
}
|
|
|
|
# GOOD — re-resolves via resolver every 10 s
|
|
location /api/ {
|
|
set $api http://api:8000;
|
|
proxy_pass $api;
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Wrong service name for docker compose exec
|
|
|
|
The API service is named `api` in `docker-compose.yml`, not `backend`.
|
|
|
|
```bash
|
|
# Wrong
|
|
docker compose exec backend alembic upgrade head
|
|
|
|
# Correct
|
|
docker compose exec api alembic upgrade head
|
|
```
|
|
|
|
---
|
|
|
|
## Alembic migration not applied after rebuild
|
|
|
|
If a new migration file was added after the last image build, the API container won't have it baked in. The container runs `alembic upgrade head` at startup from the built image.
|
|
|
|
**Fix**: rebuild the API image so the new migration file is included, then restart:
|
|
|
|
```bash
|
|
docker compose build api && docker compose up -d api
|
|
```
|
|
|
|
---
|
|
|
|
## Wrong postgres user
|
|
|
|
The database superuser is `congress` (set via `POSTGRES_USER` in `.env` / `docker-compose.yml`), not the default `postgres`.
|
|
|
|
```bash
|
|
# Wrong
|
|
docker compose exec postgres psql -U postgres pocketveto
|
|
|
|
# Correct
|
|
docker compose exec postgres psql -U congress pocketveto
|
|
```
|
|
|
|
---
|
|
|
|
## Frontend changes not showing after editing source files
|
|
|
|
The frontend runs as a production Next.js build (`NODE_ENV=production`) — there is no hot reload. Code changes require a full image rebuild:
|
|
|
|
```bash
|
|
docker compose build frontend && docker compose up -d frontend
|
|
```
|
|
|
|
Static assets are cache-busted automatically by Next.js (content-hashed filenames), so a hard refresh in the browser is not required after the new container starts.
|
|
|
|
---
|
|
|
|
## Celery tasks not reflecting code changes
|
|
|
|
Celery worker and beat processes also run from the built image. After changing any worker code:
|
|
|
|
```bash
|
|
docker compose build worker beat && docker compose up -d worker beat
|
|
```
|
|
|
|
---
|
|
|
|
## Checking logs
|
|
|
|
```bash
|
|
# All services
|
|
docker compose logs -f
|
|
|
|
# Single service (last 50 lines)
|
|
docker compose logs --tail=50 api
|
|
docker compose logs --tail=50 nginx
|
|
docker compose logs --tail=50 worker
|
|
|
|
# Follow in real time
|
|
docker compose logs -f api worker
|
|
```
|
|
|
|
---
|
|
|
|
## Inspecting the database
|
|
|
|
```bash
|
|
docker compose exec postgres psql -U congress pocketveto
|
|
```
|
|
|
|
Useful queries:
|
|
|
|
```sql
|
|
-- Recent notification events
|
|
SELECT event_type, bill_id, dispatched_at, created_at
|
|
FROM notification_events
|
|
ORDER BY created_at DESC
|
|
LIMIT 20;
|
|
|
|
-- Follow modes per user
|
|
SELECT u.email, f.follow_type, f.follow_value, f.follow_mode
|
|
FROM follows f
|
|
JOIN users u ON u.id = f.user_id
|
|
ORDER BY u.email, f.follow_type;
|
|
|
|
-- Users and their RSS tokens
|
|
SELECT id, email, rss_token IS NOT NULL AS has_rss_token FROM users;
|
|
```
|