hoopscout/README.md

# HoopScout

HoopScout is a production-minded basketball scouting and player search platform.
The main product experience is server-rendered Django Templates with HTMX enhancements.
A minimal read-only API is included as a secondary integration surface.

## Core Stack

- Python 3.12+
- Django
- Django Templates + HTMX
- Tailwind CSS (CLI build pipeline)
- PostgreSQL
- Redis
- Celery + Celery Beat
- Django REST Framework (read-only API)
- pytest
- Docker / Docker Compose
- nginx

## Architecture Summary

- Main UI: Django + HTMX (not SPA)
- Data layer: normalized domain models for players, seasons, competitions, teams, stats, scouting state
- Provider integration: adapter-based abstraction in `apps/providers`
- Ingestion orchestration: `apps/ingestion` with run/error logs and Celery task execution
- Optional API: read-only DRF endpoints under `/api/`

## Repository Structure

```text
.
├── apps/
│   ├── api/
│   ├── competitions/
│   ├── core/
│   ├── ingestion/
│   ├── players/
│   ├── providers/
│   ├── scouting/
│   ├── stats/
│   ├── teams/
│   └── users/
├── config/
│   └── settings/
├── docs/
├── nginx/
├── requirements/
├── package.json
├── tailwind.config.js
├── static/
├── templates/
├── tests/
├── .github/
├── CHANGELOG.md
├── docker-compose.yml
├── Dockerfile
└── entrypoint.sh
```

## Quick Start

1. Create local env file:

```bash
cp .env.example .env
```

2. Build and run services:

```bash
docker compose up --build
```

3. If `AUTO_APPLY_MIGRATIONS=0`, run migrations manually:

```bash
docker compose exec web python manage.py migrate
```

4. Create a superuser:

```bash
docker compose exec web python manage.py createsuperuser
```

5. Open the app:

- Web: http://localhost
- Admin: http://localhost/admin/
- Health: http://localhost/health/
- API root endpoints: `/api/players/`, `/api/competitions/`, `/api/teams/`, `/api/seasons/`

## Setup and Run Notes

- `web` service starts through `entrypoint.sh` and waits for PostgreSQL readiness.
- `web` service also builds Tailwind CSS before `collectstatic` when `AUTO_BUILD_TAILWIND=1`.
- `web`, `celery_worker`, `celery_beat`, and `tailwind` run as a non-root user inside the image.
- `celery_worker` executes background sync work.
- `celery_beat` triggers periodic provider sync (`apps.ingestion.tasks.scheduled_provider_sync`).
- `tailwind` service runs watch mode for development (`npm run dev`).
- nginx proxies web traffic and serves static/media volume mounts.

## Docker Volumes and Persistence

`docker-compose.yml` uses named volumes:

- `postgres_data`: PostgreSQL persistent database
- `static_data`: collected static assets
- `media_data`: user/provider media artifacts
- `runtime_data`: app runtime files (e.g., celery beat schedule)
- `redis_data`: Redis persistence (`/data` for RDB/AOF files)
- `node_modules_data`: Node modules cache for Tailwind builds in containers

This keeps persistent state outside container lifecycles.

## Migrations

Create migration files:

```bash
docker compose exec web python manage.py makemigrations
```

Apply migrations:

```bash
docker compose exec web python manage.py migrate
```

## Testing

Run all tests:

```bash
docker compose run --rm web sh -lc 'pip install -r requirements/dev.txt && pytest -q'
```

Run a focused module:

```bash
docker compose run --rm web sh -lc 'pip install -r requirements/dev.txt && pytest -q tests/test_api.py'
```

## Frontend Assets (Tailwind)

Build Tailwind once:

```bash
docker compose run --rm web sh -lc 'npm install --no-audit --no-fund && npm run build'
```

Run Tailwind in watch mode during development:

```bash
docker compose up tailwind
```

Source CSS lives in `static/src/tailwind.css` and compiles to `static/css/main.css`.
HTMX is served from local static assets (`static/vendor/htmx.min.js`) instead of a CDN dependency.

## Production Configuration

Use production settings in deployed environments:

```bash
DJANGO_SETTINGS_MODULE=config.settings.production
DJANGO_DEBUG=0
DJANGO_ENV=production
```

When `DJANGO_DEBUG=0`, startup fails fast unless:

- `DJANGO_SECRET_KEY` is a real non-default value
- `DJANGO_ALLOWED_HOSTS` is set
- `DJANGO_CSRF_TRUSTED_ORIGINS` is set (for production settings)

Production settings enable hardened defaults such as:

- secure cookies
- HSTS
- security headers
- `ManifestStaticFilesStorage` for static asset integrity/versioning

## Superuser and Auth

Create superuser:

```bash
docker compose exec web python manage.py createsuperuser
```

Default auth routes:

- Signup: `/users/signup/`
- Login: `/users/login/`
- Logout: `/users/logout/`

## Ingestion and Manual Sync

### Trigger via Django Admin

- Open `/admin/` -> `IngestionRun`
- Use admin actions:
  - `Queue full sync (default provider)`
  - `Queue incremental sync (default provider)`
  - `Retry selected ingestion runs`

### Trigger from shell (manual)

```bash
docker compose exec web python manage.py shell
```

```python
from apps.ingestion.tasks import trigger_full_sync
trigger_full_sync.delay(provider_namespace="balldontlie")
```

### Logs and diagnostics

- Run-level status/counters: `IngestionRun`
- Structured error records: `IngestionError`
- Provider entity mappings + diagnostic payload snippets: `ExternalMapping`
- `IngestionRun.error_summary` captures top-level failure/partial-failure context

### Scheduled sync via Celery Beat

Configure scheduled sync through environment variables:

- `INGESTION_SCHEDULE_ENABLED` (`0`/`1`)
- `INGESTION_SCHEDULE_CRON` (5-field cron expression, default `*/30 * * * *`)
- `INGESTION_SCHEDULE_PROVIDER_NAMESPACE` (optional; falls back to default provider namespace)
- `INGESTION_SCHEDULE_JOB_TYPE` (`incremental` or `full_sync`)
- `INGESTION_PREVENT_OVERLAP` (`0`/`1`) to skip obvious overlapping runs
- `INGESTION_OVERLAP_WINDOW_MINUTES` overlap guard window

When enabled, Celery Beat enqueues the scheduled sync task on the configured cron.
The task uses the existing ingestion service path and writes run/error records in the same tables as manual sync.

## Provider Backend Selection

Provider backend is selected via environment variables:

- `PROVIDER_BACKEND=demo` uses the local JSON fixture adapter (`mvp_demo`)
- `PROVIDER_BACKEND=balldontlie` uses the HTTP adapter (`balldontlie`)
- `PROVIDER_DEFAULT_NAMESPACE` can override backend mapping explicitly

The balldontlie adapter is NBA-centric and intended as MVP ingestion only. The provider abstraction remains ready for future multi-league providers (for example Sportradar or FIBA GDAP).

## GitFlow Workflow

GitFlow is required in this repository:

- `main`: production branch
- `develop`: integration branch
- `feature/*`: new feature branches from `develop`
- `release/*`: release hardening branches from `develop`
- `hotfix/*`: urgent production fixes from `main`

Read full details in [CONTRIBUTING.md](CONTRIBUTING.md) and [docs/workflow.md](docs/workflow.md).

### Repository Bootstrap Commands

Run these from the current `main` branch to initialize local GitFlow usage:

```bash
git checkout main
git pull origin main
git checkout -b develop
git push -u origin develop
```

Start a feature branch:

```bash
git checkout develop
git pull origin develop
git checkout -b feature/player-search-tuning
```

Start a release branch:

```bash
git checkout develop
git pull origin develop
git checkout -b release/0.1.0
```

Start a hotfix branch:

```bash
git checkout main
git pull origin main
git checkout -b hotfix/fix-redis-persistence
```

## Release Notes / Changelog Convention

- Use [CHANGELOG.md](CHANGELOG.md) with an `Unreleased` section.
- For each merged PR, add short entries under:
  - `Added`
  - `Changed`
  - `Fixed`
- On release, move `Unreleased` items to a dated version section (`[x.y.z] - YYYY-MM-DD`).