# Runbook

## Installation screen remains visible

1. Check `docker compose ps` for healthy database and WordPress services.
2. Run `docker compose --profile tools run --rm wp-cli /scripts/bootstrap.sh`.
3. Inspect `docker compose logs wordpress db` without printing secret values.

## WordPress stays in waiting or unhealthy

1. Inspect `docker compose logs wordpress db` without printing secret values.
2. Confirm the database volume was initialized with the same `MARIADB_DATABASE`,
   `MARIADB_USER`, and password currently configured in `.env`.
3. Confirm `WP_URL` has the intended scheme and hostname. In production it should be
   the public HTTPS URL.
4. Rebuild after changes to the WordPress image:

   ```bash
   docker compose up --build -d wordpress
   ```

The WordPress healthcheck is internal and does not depend on the public DNS/TLS route.
It sends the configured host and forwarded protocol headers to avoid following external
HTTPS redirects during production startup.

## NGINX returns 502

1. Run `docker compose ps` and `docker compose exec proxy nginx -t`.
2. Check the WordPress health status and `docker compose logs proxy wordpress`.
3. Confirm the request host is exactly `azionelab.org`; unknown hosts return 404.

## Images or theme are missing

1. Confirm `WORDPRESS_DATA_PATH` is mounted in both WordPress and WP-CLI.
2. Run `docker compose --profile tools run --rm wp-cli -c 'wp theme status azionelab'`.
3. Verify file ownership before changing permissions; never make the tree world-writable.

## WP-CLI cannot resolve wordpress.org

The default WP-CLI container intentionally runs only on the internal data network. Use
the maintenance override for commands that must download WordPress core files:

```bash
docker compose -f docker-compose.yml -f docker-compose.maintenance.yml --profile tools run --rm wp-cli -c 'wp core update --version=7.0.2 --force'
docker compose -f docker-compose.yml -f docker-compose.maintenance.yml --profile tools run --rm wp-cli -c 'wp core update-db'
docker compose --profile tools run --rm wp-cli -c 'wp core version'
docker compose restart wordpress proxy
```

If DNS still fails with the maintenance override, inspect the host/container DNS
configuration and firewall rules before retrying the update.

## Scheduled WordPress events are late

The public `/wp-cron.php` trigger is intentionally blocked. Scheduled jobs are executed
by the internal `wp-cron` service.

1. Confirm the runner is present:

   ```bash
   docker compose ps wp-cron
   ```

2. Inspect due events:

   ```bash
   docker compose --profile tools run --rm wp-cli -c 'wp cron event list'
   ```

3. Inspect runner logs without printing `.env` values:

   ```bash
   docker compose logs --tail=100 wp-cron wordpress db
   ```

4. If needed, run due events manually:

   ```bash
   docker compose --profile tools run --rm wp-cli -c 'wp cron event run --due-now'
   ```

The Site Health loopback test may still report a blocked public loopback while the edge
continues to deny `/wp-cron.php`; that is an expected trade-off of the hardened public
configuration.

## Uploaded image does not appear on the public page

1. Confirm the image was selected in **Appearance > Customize**, not only uploaded in
   **Media**. For the teacher portrait, use **Appearance > Customize > Il maestro >
   Foto**, then publish the change.
2. Hard-refresh the browser or purge the CDN cache. An NGINX log status `304` for
   `/wp-content/uploads/...` is not an application error; it means the browser already
   has a cached copy and NGINX did not resend the file body.
3. If the upload appears in Media but not on the page, inspect the generated `<img>`
   URL and verify it uses the public scheme/host configured by `WP_URL`.
4. Confirm the uploaded file exists under `WORDPRESS_DATA_PATH/wp-content/uploads` and
   is readable by the WordPress container user.

## Show card description is hard to find

Edit **Spettacoli**, open the show, then use **Dettagli spettacolo > Descrizione
breve**. That field is rendered below the title in the homepage show card and is stored
as the standard WordPress excerpt.

The small uppercase line above the title is not the description. It is built from
**Dettagli spettacolo > Anno** and **Luogo**. Empty fields are hidden on the public
card.

Shows and gallery items intentionally use the classic WordPress editor so the metadata
box remains visible and predictable. If the block editor appears for these objects,
rebuild/recreate the WordPress container so the current must-use plugin is copied into
the persistent `WORDPRESS_DATA_PATH` volume.

## A service cannot write to its volume

1. Stop the affected service.
2. Confirm the relevant `.env` path points to the intended host directory.
3. Run `./scripts/prepare-host-volumes.sh`.
4. Start the service and inspect logs without printing secret values.

## Certificate issuance fails

Verify public DNS, inbound port 80, the operator email, and staging mode. Request a
missing `/.well-known/acme-challenge/` path: an NGINX 404 confirms the route is yours.
Avoid repeated production-CA retries while debugging.

When Let's Encrypt is enabled, the proxy intentionally returns 503 for normal
application paths until a certificate exists. ACME challenge paths and proxy health
paths must still work in that pending state, otherwise Certbot will never start.

## Staging certificate remains after switching to production

Certbot does not replace a still-valid staging certificate just because
`LETSENCRYPT_STAGING` changed. Recreate the Certbot container so it reads the updated
environment:

```bash
docker compose up -d --force-recreate certbot
docker compose up -d proxy
```

The entrypoint removes an existing staging certificate for `LETSENCRYPT_DOMAIN` before
requesting the production certificate. If issuance fails, inspect `docker compose logs
certbot proxy`, verify DNS and port 80, and avoid repeated production-CA retries.

## Bootstrap writes to the wrong data volume

Do not combine the test override with production bootstrap commands. This command writes
to isolated test volumes only:

```bash
docker compose -f docker-compose.yml -f docker-compose.test.yml --profile tools run --rm wp-cli /scripts/bootstrap.sh
```

Use this command for the real host-based WordPress data directory:

```bash
docker compose --profile tools run --rm wp-cli /scripts/bootstrap.sh
```

## Rollback

Revert the deployment commit and rebuild while preserving all host data directories.
Restore database/files only for a data rollback and only from a verified coordinated
backup.