PocketVeto/TROUBLESHOOTING.md

# Troubleshooting

Common issues encountered during development and deployment of PocketVeto.

---

## 502 Bad Gateway after rebuilding a container

**Symptom**

All API calls return 502. nginx error log shows:

```
connect() failed (111: Connection refused) while connecting to upstream,
upstream: "http://172.18.0.X:8000/api/..."
```

The IP in the error is the *old* IP of the container before the rebuild.

**Root cause**

When nginx uses `upstream` blocks, it resolves hostnames once at process startup and caches the result for the lifetime of the process. Rebuilding a container (e.g. `docker compose build api && docker compose up -d api`) assigns it a new Docker network IP. nginx still holds the old IP and all connections are refused.

**Immediate fix**

```bash
docker compose restart nginx
```

This forces nginx to re-resolve all upstream hostnames from Docker's internal DNS (`127.0.0.11`).

**Permanent fix (already applied)**

Replace `upstream` blocks with `set $variable` in `proxy_pass`. nginx only activates the `resolver` directive when a variable is used — making it re-resolve on each request cycle (every `valid=N` seconds).

```nginx
resolver 127.0.0.11 valid=10s ipv6=off;

# BAD — resolves once at startup, caches forever
upstream api {
    server api:8000;
}
location /api/ {
    proxy_pass http://api;
}

# GOOD — re-resolves via resolver every 10 s
location /api/ {
    set $api http://api:8000;
    proxy_pass $api;
}
```

---

## Wrong service name for docker compose exec

The API service is named `api` in `docker-compose.yml`, not `backend`.

```bash
# Wrong
docker compose exec backend alembic upgrade head

# Correct
docker compose exec api alembic upgrade head
```

---

## Alembic migration not applied after rebuild

If a new migration file was added after the last image build, the API container won't have it baked in. The container runs `alembic upgrade head` at startup from the built image.

**Fix**: rebuild the API image so the new migration file is included, then restart:

```bash
docker compose build api && docker compose up -d api
```

---

## Wrong postgres user

The database superuser is `congress` (set via `POSTGRES_USER` in `.env` / `docker-compose.yml`), not the default `postgres`.

```bash
# Wrong
docker compose exec postgres psql -U postgres pocketveto

# Correct
docker compose exec postgres psql -U congress pocketveto
```

---

## Frontend changes not showing after editing source files

The frontend runs as a production Next.js build (`NODE_ENV=production`) — there is no hot reload. Code changes require a full image rebuild:

```bash
docker compose build frontend && docker compose up -d frontend
```

Static assets are cache-busted automatically by Next.js (content-hashed filenames), so a hard refresh in the browser is not required after the new container starts.

---

## Celery tasks not reflecting code changes

Celery worker and beat processes also run from the built image. After changing any worker code:

```bash
docker compose build worker beat && docker compose up -d worker beat
```

---

## Checking logs

```bash
# All services
docker compose logs -f

# Single service (last 50 lines)
docker compose logs --tail=50 api
docker compose logs --tail=50 nginx
docker compose logs --tail=50 worker

# Follow in real time
docker compose logs -f api worker
```

---

## Inspecting the database

```bash
docker compose exec postgres psql -U congress pocketveto
```

Useful queries:

```sql
-- Recent notification events
SELECT event_type, bill_id, dispatched_at, created_at
FROM notification_events
ORDER BY created_at DESC
LIMIT 20;

-- Follow modes per user
SELECT u.email, f.follow_type, f.follow_value, f.follow_mode
FROM follows f
JOIN users u ON u.id = f.user_id
ORDER BY u.email, f.follow_type;

-- Users and their RSS tokens
SELECT id, email, rss_token IS NOT NULL AS has_rss_token FROM users;
```