azionelab-v2/docs/runbook.md

Runbook

Site shows fallback content

1. Check docker compose ps and the backend health status.
2. Request http://azionelab.org:8080/api/site/home/ through NGINX.
3. Inspect docker compose logs backend db.
4. Verify PUBLIC_CMS_API_URL resolves from the frontend container.
5. Restart the frontend after recovery so its static page is rebuilt from CMS data.

Wagtail does not start

1. Confirm PostgreSQL is healthy.
2. Check DATABASE_URL and allowed hosts without printing secret values in shared logs.
3. Run docker compose exec backend python manage.py migrate.
4. Review backend logs for a specific migration or configuration error.

Images are missing

1. Verify the media_data volume is mounted at /app/media.
2. Check WAGTAILADMIN_BASE_URL uses a URL reachable by the browser.
3. Confirm the image still exists in Wagtail and has not been removed from the volume.

A service is unhealthy

1. Run docker compose ps and inspect docker compose logs --tail=200 SERVICE.
2. For PostgreSQL, check volume capacity and database/user configuration.
3. For the backend, check /health/, migrations, and database connectivity.
4. For the frontend, request port 4321, then check the API separately because fallback content can mask a CMS outage.

NGINX returns 404 or 502

1. Confirm the request host is exactly azionelab.org; unknown hosts intentionally receive 404.
2. Run docker compose exec proxy nginx -t.
3. Check docker compose ps and verify both backend and frontend are healthy.
4. Inspect docker compose logs --tail=200 proxy backend frontend.
5. Verify local DNS or /etc/hosts maps azionelab.org to the proxy address.

Let's Encrypt does not issue a certificate

1. Confirm LETSENCRYPT_ENABLED=1 and that docker compose ps -a certbot proxy shows both containers running and the proxy healthy.
2. Check that the domain's public A/AAAA records point to this host. Remove an AAAA record if IPv6 does not actually reach it.
3. Verify inbound TCP port 80 reaches NGINX; HTTP-01 cannot use only port 443.
4. Request http://azionelab.org/.well-known/acme-challenge/missing: a 404 from NGINX confirms the challenge route is reachable, while a timeout or another server does not.
5. Inspect docker compose logs --tail=200 certbot proxy for ACME validation or rate limit errors. Do not repeatedly retry the production CA; use staging while fixing connectivity.

HTTPS is not activated after issuance

1. Run docker compose run --rm --no-deps --entrypoint certbot certbot certificates.
2. Confirm LETSENCRYPT_DOMAIN exactly matches the certificate name used by both services.
3. Wait for TLS_RELOAD_INTERVAL_SECONDS, then inspect proxy logs for nginx -t or reload errors.
4. Run docker compose exec proxy nginx -t and check HTTPS locally with an explicit DNS override.

Rollback

Disable Certbot with LETSENCRYPT_ENABLED=0 if TLS is moving to a load balancer, then recreate the affected services. Revert the application commit and rebuild containers for a full rollback. Preserve database, media, and certificate volumes. Before reversing migrations or deleting volumes, make and validate coordinated backups.