generated from bisco/codex-bootstrap
bisco
71 lines
3.1 KiB
Markdown
71 lines
3.1 KiB
Markdown
# Runbook
|
|
|
|
## Site shows fallback content
|
|
|
|
1. Check `docker compose ps` and the backend health status.
|
|
2. Request `http://azionelab.org:8080/api/site/home/` through NGINX.
|
|
3. Inspect `docker compose logs backend db`.
|
|
4. Verify `PUBLIC_CMS_API_URL` resolves from the frontend container.
|
|
5. Restart the frontend after recovery so its static page is rebuilt from CMS data.
|
|
|
|
## Wagtail does not start
|
|
|
|
1. Confirm PostgreSQL is healthy.
|
|
2. Check `DATABASE_URL` and allowed hosts without printing secret values in shared logs.
|
|
3. Run `docker compose exec backend python manage.py migrate`.
|
|
4. Review backend logs for a specific migration or configuration error.
|
|
|
|
## Images are missing
|
|
|
|
1. Verify the `media_data` volume is mounted at `/app/media`.
|
|
2. Check `WAGTAILADMIN_BASE_URL` uses a URL reachable by the browser.
|
|
3. Confirm the image still exists in Wagtail and has not been removed from the volume.
|
|
|
|
## A service is unhealthy
|
|
|
|
1. Run `docker compose ps` and inspect `docker compose logs --tail=200 SERVICE`.
|
|
2. For PostgreSQL, check volume capacity and database/user configuration.
|
|
3. For the backend, check `/health/`, migrations, and database connectivity.
|
|
4. For the frontend, request port `4321`, then check the API separately because
|
|
fallback content can mask a CMS outage.
|
|
|
|
## NGINX returns 404 or 502
|
|
|
|
1. Confirm the request host is exactly `azionelab.org`; unknown hosts intentionally
|
|
receive 404.
|
|
2. Run `docker compose exec proxy nginx -t`.
|
|
3. Check `docker compose ps` and verify both backend and frontend are healthy.
|
|
4. Inspect `docker compose logs --tail=200 proxy backend frontend`.
|
|
5. Verify local DNS or `/etc/hosts` maps `azionelab.org` to the proxy address.
|
|
|
|
## Let's Encrypt does not issue a certificate
|
|
|
|
1. Confirm `LETSENCRYPT_ENABLED=1` and that `docker compose ps -a certbot proxy` shows
|
|
both containers running and the proxy healthy.
|
|
2. Check that the domain's public A/AAAA records point to this host. Remove an AAAA
|
|
record if IPv6 does not actually reach it.
|
|
3. Verify inbound TCP port 80 reaches NGINX; HTTP-01 cannot use only port 443.
|
|
4. Request `http://azionelab.org/.well-known/acme-challenge/missing`: a 404 from NGINX
|
|
confirms the challenge route is reachable, while a timeout or another server does
|
|
not.
|
|
5. Inspect `docker compose logs --tail=200 certbot proxy` for ACME validation or rate
|
|
limit errors. Do not repeatedly retry the production CA; use staging while fixing
|
|
connectivity.
|
|
|
|
## HTTPS is not activated after issuance
|
|
|
|
1. Run `docker compose run --rm --no-deps --entrypoint certbot certbot certificates`.
|
|
2. Confirm `LETSENCRYPT_DOMAIN` exactly matches the certificate name used by both
|
|
services.
|
|
3. Wait for `TLS_RELOAD_INTERVAL_SECONDS`, then inspect proxy logs for `nginx -t` or
|
|
reload errors.
|
|
4. Run `docker compose exec proxy nginx -t` and check HTTPS locally with an explicit
|
|
DNS override.
|
|
|
|
## Rollback
|
|
|
|
Disable Certbot with `LETSENCRYPT_ENABLED=0` if TLS is moving to a load balancer, then
|
|
recreate the affected services. Revert the application commit and rebuild containers
|
|
for a full rollback. Preserve database, media, and certificate volumes. Before
|
|
reversing migrations or deleting volumes, make and validate coordinated backups.
|