Merge branch 'feature/v2-docs-runtime-consistency-audit' into feature/hoopscout-v2-static-architecture
This commit is contained in:
@ -45,6 +45,13 @@ docker compose -f docker-compose.yml -f docker-compose.dev.yml up --build
|
||||
docker compose -f docker-compose.yml -f docker-compose.release.yml up -d --build
|
||||
```
|
||||
|
||||
### Verify release topology assumptions
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.yml -f docker-compose.release.yml config
|
||||
./scripts/verify_release_topology.sh
|
||||
```
|
||||
|
||||
## Day-to-Day Feature Workflow
|
||||
|
||||
1. Sync `develop`
|
||||
@ -88,6 +95,7 @@ docker compose -f docker-compose.yml -f docker-compose.dev.yml run --rm web sh -
|
||||
- Keep snapshot storage file-based and volume-backed.
|
||||
- Do not introduce MongoDB or Elasticsearch as source of truth.
|
||||
- Keep legacy provider/Celery sync code isolated behind `LEGACY_PROVIDER_STACK_ENABLED=1`.
|
||||
- Keep runtime/docs consistency aligned with `docs/runtime-consistency-checklist.md`.
|
||||
|
||||
## Repository Bootstrap Commands
|
||||
|
||||
|
||||
46
README.md
46
README.md
@ -9,9 +9,8 @@ Current v2 foundation scope in this branch:
|
||||
- management-command-driven runtime operations
|
||||
- static snapshot directories persisted via Docker named volumes
|
||||
- strict JSON snapshot schema + import management command
|
||||
|
||||
Out of scope in this step:
|
||||
- extractor implementation
|
||||
- extractor framework with LBA/BCL/public JSON adapters
|
||||
- daily orchestration command and optional scheduler profile
|
||||
|
||||
## Runtime Architecture (v2)
|
||||
|
||||
@ -48,6 +47,7 @@ Reserved for future optional scheduler use:
|
||||
- `docker-compose.yml`: production-minded baseline runtime (immutable image filesystem)
|
||||
- `docker-compose.dev.yml`: development override with source bind mount for `web`
|
||||
- `docker-compose.release.yml`: production settings override (`DJANGO_SETTINGS_MODULE=config.settings.production`)
|
||||
- `scripts/verify_release_topology.sh`: validates merged release compose has no source-code bind mounts for runtime services
|
||||
|
||||
### Start development runtime
|
||||
|
||||
@ -74,6 +74,31 @@ For development override:
|
||||
docker compose -f docker-compose.yml -f docker-compose.dev.yml --profile scheduler up -d scheduler
|
||||
```
|
||||
|
||||
### Runtime Modes At A Glance
|
||||
|
||||
- development (`docker-compose.yml` + `docker-compose.dev.yml`):
|
||||
- mutable source bind mounts for `web` and `scheduler`
|
||||
- optimized for local iteration
|
||||
- release-style (`docker-compose.yml` + `docker-compose.release.yml`):
|
||||
- immutable app filesystem for runtime services
|
||||
- production settings enabled for Django
|
||||
- scheduler profile:
|
||||
- only starts when `--profile scheduler` is used
|
||||
- if started with `SCHEDULER_ENABLED=0`, scheduler stays in idle sleep mode (no restart loop exit behavior)
|
||||
|
||||
### Release Topology Verification
|
||||
|
||||
Verify merged release config and immutability:
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.yml -f docker-compose.release.yml config
|
||||
./scripts/verify_release_topology.sh
|
||||
```
|
||||
|
||||
Verification expectation:
|
||||
- `web` and `scheduler` must not bind-mount repository source code in release mode.
|
||||
- named volumes for DB/static/media/snapshots remain mounted.
|
||||
|
||||
## Named Volumes
|
||||
|
||||
v2 runtime uses named volumes for persistence:
|
||||
@ -86,6 +111,11 @@ v2 runtime uses named volumes for persistence:
|
||||
|
||||
Development override uses separate dev-prefixed volumes to avoid ownership collisions.
|
||||
|
||||
Snapshot volume intent:
|
||||
- `snapshots_incoming`: extractor output waiting for import
|
||||
- `snapshots_archive`: successfully imported files
|
||||
- `snapshots_failed`: schema/processing failures for operator inspection
|
||||
|
||||
## Environment Variables
|
||||
|
||||
Use `.env.example` as the source of truth.
|
||||
@ -99,6 +129,9 @@ Core groups:
|
||||
- daily orchestration vars (`DAILY_ORCHESTRATION_*`)
|
||||
- optional legacy provider-sync toggle (`LEGACY_PROVIDER_STACK_ENABLED`)
|
||||
|
||||
Operational reference:
|
||||
- `docs/runtime-consistency-checklist.md`
|
||||
|
||||
## Snapshot Storage Convention
|
||||
|
||||
Snapshot files are expected under:
|
||||
@ -199,6 +232,12 @@ Command behavior:
|
||||
- moves valid files to archive
|
||||
- moves invalid files to failed
|
||||
|
||||
Import lifecycle summary:
|
||||
1. extractor writes normalized snapshots to `incoming`
|
||||
2. `import_snapshots` validates + upserts to PostgreSQL
|
||||
3. imported files move to `archive`
|
||||
4. invalid files move to `failed` with error details in `ImportFile`
|
||||
|
||||
### Source Identity Namespacing
|
||||
|
||||
Raw external IDs are **not globally unique** across basketball data sources. HoopScout v2 uses a namespaced identity for imported entities:
|
||||
@ -338,6 +377,7 @@ Notes:
|
||||
## Testing
|
||||
|
||||
- runtime `web` image stays lean and may not include `pytest` tooling
|
||||
- runtime containers (`web`/`nginx`/`scheduler`) are for serving/orchestration, not preloaded test tooling
|
||||
- run tests with the development compose stack (or a dedicated test image/profile) and install dev dependencies first
|
||||
- local example (one-off):
|
||||
|
||||
|
||||
58
docs/runtime-consistency-checklist.md
Normal file
58
docs/runtime-consistency-checklist.md
Normal file
@ -0,0 +1,58 @@
|
||||
# Runtime Consistency Checklist (v2)
|
||||
|
||||
Use this checklist when runtime/docs changes are made.
|
||||
|
||||
## Compose and Runtime
|
||||
|
||||
- `docker-compose.yml` contains only v2 default runtime services:
|
||||
- `web`, `nginx`, `postgres`
|
||||
- optional `scheduler` profile service
|
||||
- `docker-compose.dev.yml` is mutable (source bind mounts allowed for dev only).
|
||||
- `docker-compose.release.yml` is settings-focused and keeps release runtime immutable.
|
||||
|
||||
## Image/Registry Strategy
|
||||
|
||||
- `web` image: `registry.younerd.org/hoopscout/web:${APP_IMAGE_TAG:-latest}`
|
||||
- `nginx` image: `registry.younerd.org/hoopscout/nginx:${NGINX_IMAGE_TAG:-latest}`
|
||||
- optional scheduler image: `registry.younerd.org/hoopscout/scheduler:${APP_IMAGE_TAG:-latest}`
|
||||
|
||||
## Entrypoints
|
||||
|
||||
- `entrypoint.sh`:
|
||||
- waits for PostgreSQL
|
||||
- creates snapshot directories
|
||||
- optionally runs `migrate` and `collectstatic` when booting gunicorn
|
||||
- `scripts/scheduler.sh`:
|
||||
- runs `run_daily_orchestration` loop
|
||||
- idle-sleeps when `SCHEDULER_ENABLED=0`
|
||||
|
||||
## Snapshot Lifecycle
|
||||
|
||||
1. Extractor writes snapshots to `incoming`.
|
||||
2. `import_snapshots` validates + upserts into PostgreSQL.
|
||||
3. Success => file moved to `archive`.
|
||||
4. Failure => file moved to `failed`.
|
||||
|
||||
## Source Identity Rule
|
||||
|
||||
Raw IDs are not global. Imported identities are namespaced by source:
|
||||
|
||||
- `Competition`: `(source_name, source_uid)`
|
||||
- `Team`: `(source_name, source_uid)`
|
||||
- `Player`: `(source_name, source_uid)`
|
||||
|
||||
## Legacy Isolation
|
||||
|
||||
- `LEGACY_PROVIDER_STACK_ENABLED=0` by default.
|
||||
- With default setting:
|
||||
- `apps.providers` is not installed
|
||||
- `/providers/` routes are not mounted
|
||||
- legacy provider settings are not required
|
||||
|
||||
## Verification Commands
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.yml -f docker-compose.release.yml config
|
||||
./scripts/verify_release_topology.sh
|
||||
docker compose -f docker-compose.yml -f docker-compose.dev.yml run --rm web sh -lc "export PYTHONUSERBASE=/tmp/pyuser && python -m pip install --user -r requirements/dev.txt && python -m pytest -q"
|
||||
```
|
||||
@ -30,7 +30,6 @@ check_service_bind_mount() {
|
||||
}
|
||||
|
||||
check_service_bind_mount "web"
|
||||
check_service_bind_mount "celery_worker"
|
||||
check_service_bind_mount "celery_beat"
|
||||
check_service_bind_mount "scheduler"
|
||||
|
||||
echo "Release topology verification passed."
|
||||
|
||||
Reference in New Issue
Block a user