From 887da3cd064e3d4191bf7863a7b908abd2dc4237 Mon Sep 17 00:00:00 2001 From: Alfredo Di Stasio Date: Fri, 20 Mar 2026 16:02:12 +0100 Subject: [PATCH] docs(v2): align runtime and operations documentation with compose behavior --- CONTRIBUTING.md | 8 ++++ README.md | 46 +++++++++++++++++++-- docs/runtime-consistency-checklist.md | 58 +++++++++++++++++++++++++++ scripts/verify_release_topology.sh | 3 +- 4 files changed, 110 insertions(+), 5 deletions(-) create mode 100644 docs/runtime-consistency-checklist.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 59712cf..5fe6989 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -45,6 +45,13 @@ docker compose -f docker-compose.yml -f docker-compose.dev.yml up --build docker compose -f docker-compose.yml -f docker-compose.release.yml up -d --build ``` +### Verify release topology assumptions + +```bash +docker compose -f docker-compose.yml -f docker-compose.release.yml config +./scripts/verify_release_topology.sh +``` + ## Day-to-Day Feature Workflow 1. Sync `develop` @@ -88,6 +95,7 @@ docker compose -f docker-compose.yml -f docker-compose.dev.yml run --rm web sh - - Keep snapshot storage file-based and volume-backed. - Do not introduce MongoDB or Elasticsearch as source of truth. - Keep legacy provider/Celery sync code isolated behind `LEGACY_PROVIDER_STACK_ENABLED=1`. +- Keep runtime/docs consistency aligned with `docs/runtime-consistency-checklist.md`. ## Repository Bootstrap Commands diff --git a/README.md b/README.md index 20ab074..95714d6 100644 --- a/README.md +++ b/README.md @@ -9,9 +9,8 @@ Current v2 foundation scope in this branch: - management-command-driven runtime operations - static snapshot directories persisted via Docker named volumes - strict JSON snapshot schema + import management command - -Out of scope in this step: -- extractor implementation +- extractor framework with LBA/BCL/public JSON adapters +- daily orchestration command and optional scheduler profile ## Runtime Architecture (v2) @@ -48,6 +47,7 @@ Reserved for future optional scheduler use: - `docker-compose.yml`: production-minded baseline runtime (immutable image filesystem) - `docker-compose.dev.yml`: development override with source bind mount for `web` - `docker-compose.release.yml`: production settings override (`DJANGO_SETTINGS_MODULE=config.settings.production`) +- `scripts/verify_release_topology.sh`: validates merged release compose has no source-code bind mounts for runtime services ### Start development runtime @@ -74,6 +74,31 @@ For development override: docker compose -f docker-compose.yml -f docker-compose.dev.yml --profile scheduler up -d scheduler ``` +### Runtime Modes At A Glance + +- development (`docker-compose.yml` + `docker-compose.dev.yml`): + - mutable source bind mounts for `web` and `scheduler` + - optimized for local iteration +- release-style (`docker-compose.yml` + `docker-compose.release.yml`): + - immutable app filesystem for runtime services + - production settings enabled for Django +- scheduler profile: + - only starts when `--profile scheduler` is used + - if started with `SCHEDULER_ENABLED=0`, scheduler stays in idle sleep mode (no restart loop exit behavior) + +### Release Topology Verification + +Verify merged release config and immutability: + +```bash +docker compose -f docker-compose.yml -f docker-compose.release.yml config +./scripts/verify_release_topology.sh +``` + +Verification expectation: +- `web` and `scheduler` must not bind-mount repository source code in release mode. +- named volumes for DB/static/media/snapshots remain mounted. + ## Named Volumes v2 runtime uses named volumes for persistence: @@ -86,6 +111,11 @@ v2 runtime uses named volumes for persistence: Development override uses separate dev-prefixed volumes to avoid ownership collisions. +Snapshot volume intent: +- `snapshots_incoming`: extractor output waiting for import +- `snapshots_archive`: successfully imported files +- `snapshots_failed`: schema/processing failures for operator inspection + ## Environment Variables Use `.env.example` as the source of truth. @@ -99,6 +129,9 @@ Core groups: - daily orchestration vars (`DAILY_ORCHESTRATION_*`) - optional legacy provider-sync toggle (`LEGACY_PROVIDER_STACK_ENABLED`) +Operational reference: +- `docs/runtime-consistency-checklist.md` + ## Snapshot Storage Convention Snapshot files are expected under: @@ -199,6 +232,12 @@ Command behavior: - moves valid files to archive - moves invalid files to failed +Import lifecycle summary: +1. extractor writes normalized snapshots to `incoming` +2. `import_snapshots` validates + upserts to PostgreSQL +3. imported files move to `archive` +4. invalid files move to `failed` with error details in `ImportFile` + ### Source Identity Namespacing Raw external IDs are **not globally unique** across basketball data sources. HoopScout v2 uses a namespaced identity for imported entities: @@ -338,6 +377,7 @@ Notes: ## Testing - runtime `web` image stays lean and may not include `pytest` tooling +- runtime containers (`web`/`nginx`/`scheduler`) are for serving/orchestration, not preloaded test tooling - run tests with the development compose stack (or a dedicated test image/profile) and install dev dependencies first - local example (one-off): diff --git a/docs/runtime-consistency-checklist.md b/docs/runtime-consistency-checklist.md new file mode 100644 index 0000000..ca482dc --- /dev/null +++ b/docs/runtime-consistency-checklist.md @@ -0,0 +1,58 @@ +# Runtime Consistency Checklist (v2) + +Use this checklist when runtime/docs changes are made. + +## Compose and Runtime + +- `docker-compose.yml` contains only v2 default runtime services: + - `web`, `nginx`, `postgres` + - optional `scheduler` profile service +- `docker-compose.dev.yml` is mutable (source bind mounts allowed for dev only). +- `docker-compose.release.yml` is settings-focused and keeps release runtime immutable. + +## Image/Registry Strategy + +- `web` image: `registry.younerd.org/hoopscout/web:${APP_IMAGE_TAG:-latest}` +- `nginx` image: `registry.younerd.org/hoopscout/nginx:${NGINX_IMAGE_TAG:-latest}` +- optional scheduler image: `registry.younerd.org/hoopscout/scheduler:${APP_IMAGE_TAG:-latest}` + +## Entrypoints + +- `entrypoint.sh`: + - waits for PostgreSQL + - creates snapshot directories + - optionally runs `migrate` and `collectstatic` when booting gunicorn +- `scripts/scheduler.sh`: + - runs `run_daily_orchestration` loop + - idle-sleeps when `SCHEDULER_ENABLED=0` + +## Snapshot Lifecycle + +1. Extractor writes snapshots to `incoming`. +2. `import_snapshots` validates + upserts into PostgreSQL. +3. Success => file moved to `archive`. +4. Failure => file moved to `failed`. + +## Source Identity Rule + +Raw IDs are not global. Imported identities are namespaced by source: + +- `Competition`: `(source_name, source_uid)` +- `Team`: `(source_name, source_uid)` +- `Player`: `(source_name, source_uid)` + +## Legacy Isolation + +- `LEGACY_PROVIDER_STACK_ENABLED=0` by default. +- With default setting: + - `apps.providers` is not installed + - `/providers/` routes are not mounted + - legacy provider settings are not required + +## Verification Commands + +```bash +docker compose -f docker-compose.yml -f docker-compose.release.yml config +./scripts/verify_release_topology.sh +docker compose -f docker-compose.yml -f docker-compose.dev.yml run --rm web sh -lc "export PYTHONUSERBASE=/tmp/pyuser && python -m pip install --user -r requirements/dev.txt && python -m pytest -q" +``` diff --git a/scripts/verify_release_topology.sh b/scripts/verify_release_topology.sh index 5c26490..a900e41 100755 --- a/scripts/verify_release_topology.sh +++ b/scripts/verify_release_topology.sh @@ -30,7 +30,6 @@ check_service_bind_mount() { } check_service_bind_mount "web" -check_service_bind_mount "celery_worker" -check_service_bind_mount "celery_beat" +check_service_bind_mount "scheduler" echo "Release topology verification passed."