diff --git a/docs/PHASE-index.md b/docs/PHASE-index.md index ca710d2a..2bc0ae1b 100644 --- a/docs/PHASE-index.md +++ b/docs/PHASE-index.md @@ -42,6 +42,12 @@ This document indexes all implementation phases of the ForecastLabAI project. - `app/shared/` - Shared utilities (3 modules) - `app/main.py` - FastAPI application entry point - `alembic/` - Async migration setup +- `.github/workflows/` - CI/CD pipelines (5 workflows) + - `ci.yml` - Lint, typecheck, test, migration check + - `schema-validation.yml` - Migration drift detection + - `dependency-check.yml` - Weekly vulnerability scanning + - `phase-snapshot.yml` - Audit snapshots for phase-* branches + - `cd-release.yml` - Automated semantic versioning releases **Validation Results**: - Ruff: All checks passed @@ -102,6 +108,8 @@ Each phase document (`docs/PHASE/X-PHASE_NAME.md`) contains: - [Architecture Overview](./ARCHITECTURE.md) - [ADR Index](./ADR/ADR-INDEX.md) +- [GitHub Workflows Guide](./github/github-quickstart.md) +- [GitHub Workflow Diagrams](./github/diagrams/README.md) - [Logging Standard](./validation/logging-standard.md) - [MyPy Standard](./validation/mypy-standard.md) - [Pyright Standard](./validation/pyright-standard.md) @@ -115,3 +123,4 @@ Each phase document (`docs/PHASE/X-PHASE_NAME.md`) contains: | Date | Phase | Action | |------|-------|--------| | 2026-01-26 | 0 | Initial project foundation completed | +| 2026-01-26 | 0 | Added CI/CD infrastructure (5 GitHub Actions workflows) | diff --git a/docs/PHASE/0-INIT_PHASE.md b/docs/PHASE/0-INIT_PHASE.md index ed06b313..fa717d4b 100644 --- a/docs/PHASE/0-INIT_PHASE.md +++ b/docs/PHASE/0-INIT_PHASE.md @@ -190,94 +190,18 @@ def add_request_id( #### Database (`database.py`) -Async SQLAlchemy 2.0 setup: - -```python -class Base(DeclarativeBase): - pass - -def get_engine() -> AsyncEngine: - settings = get_settings() - return create_async_engine( - settings.database_url, - echo=settings.debug, - pool_pre_ping=True, - ) - -async def get_db() -> AsyncGenerator[AsyncSession, None]: - session_maker = get_session_maker() - async with session_maker() as session: - try: - yield session - await session.commit() - except Exception: - await session.rollback() - raise -``` - -**Features**: -- Async engine with connection pool -- Session dependency for FastAPI -- Auto-commit on success, rollback on failure -- Debug mode SQL echoing +Async SQLAlchemy 2.0 with session dependency: +- `Base` - DeclarativeBase for models +- `get_engine()` - Async engine with pool_pre_ping +- `get_db()` - FastAPI dependency with auto-commit/rollback #### Middleware (`middleware.py`) -Request ID correlation middleware: - -```python -class RequestIdMiddleware(BaseHTTPMiddleware): - async def dispatch( - self, - request: Request, - call_next: Callable[[Request], Awaitable[Response]], - ) -> Response: - request_id = request.headers.get("X-Request-ID") or str(uuid.uuid4()) - token = request_id_ctx.set(request_id) - - try: - logger.info("http.request_started", method=request.method, path=str(request.url.path)) - response = await call_next(request) - logger.info("http.request_completed", status_code=response.status_code) - response.headers["X-Request-ID"] = request_id - return response - finally: - request_id_ctx.reset(token) -``` - -**Features**: -- Generates UUID if no X-Request-ID header provided -- Preserves client-provided request IDs -- Injects request ID into all log entries -- Returns request ID in response header +Request ID correlation: generates UUID if no X-Request-ID header, preserves client-provided IDs, injects into all logs, returns in response header. #### Exceptions (`exceptions.py`) -Custom exception hierarchy with handlers: - -```python -class ForecastLabError(Exception): - def __init__( - self, - message: str, - code: str = "INTERNAL_ERROR", - status_code: int = 500, - details: dict[str, Any] | None = None, - ) -> None: - self.message = message - self.code = code - self.status_code = status_code - self.details = details or {} - -class NotFoundError(ForecastLabError): - # 404 errors - -class ValidationError(ForecastLabError): - # 422 errors - -class DatabaseError(ForecastLabError): - # 500 database errors -``` +Custom hierarchy: `ForecastLabError` base class with `NotFoundError` (404), `ValidationError` (422), `DatabaseError` (500). **Error Response Format**: ```json @@ -293,36 +217,18 @@ class DatabaseError(ForecastLabError): #### Health Check (`health.py`) -Health and readiness endpoints: - | Endpoint | Purpose | Response | |----------|---------|----------| -| `GET /health` | Basic liveness check | `{"status": "ok"}` | -| `GET /health/ready` | Readiness with DB check | `{"status": "ok", "database": "connected"}` | +| `GET /health` | Liveness | `{"status": "ok"}` | +| `GET /health/ready` | Readiness + DB | `{"status": "ok", "database": "connected"}` | --- ### 4. Shared Utilities (`app/shared/`) -#### TimestampMixin (`models.py`) - -```python -class TimestampMixin: - created_at: Mapped[datetime] = mapped_column( - DateTime(timezone=True), - server_default=func.now(), - nullable=False, - ) - updated_at: Mapped[datetime] = mapped_column( - DateTime(timezone=True), - server_default=func.now(), - onupdate=func.now(), - nullable=False, - ) -``` - -#### Pagination (`schemas.py`) +- **TimestampMixin**: `created_at` and `updated_at` with server defaults +**Pagination Types** (`schemas.py`): ```python class PaginationParams(BaseModel): page: int = Field(1, ge=1) @@ -540,6 +446,108 @@ scripts/ --- +### 9. CI/CD Infrastructure (`.github/workflows/`) + +#### Core CI Pipeline (`ci.yml`) + +Runs on push/PR to main and dev branches: + +| Job | Purpose | Tools | +|-----|---------|-------| +| `lint` | Code quality | ruff check, ruff format | +| `typecheck` | Static analysis | mypy, pyright | +| `test` | Unit/integration tests | pytest with PostgreSQL | +| `migration-check` | Migration integrity | alembic upgrade head | + +**Key Features**: +- Concurrency groups to cancel stale runs +- PostgreSQL service container (pgvector:pg16) +- uv package manager with caching +- Python 3.12 environment + +#### Schema Validation (`schema-validation.yml`) + +Triggers on changes to `alembic/**` or `app/**/models.py`. + +**Steps:** +1. Fresh DB migration test (`alembic upgrade head`) +2. Migration chain integrity (single head enforcement) +3. Schema drift detection (`alembic check`) +4. Downgrade/upgrade cycle test +5. Schema report generation + +**Purpose**: Catches migration drift and schema issues before merge. + +#### Dependency Security Check (`dependency-check.yml`) + +Triggers weekly (Sunday 00:00 UTC) or manual dispatch. + +**Steps:** +1. Export requirements from uv lock +2. Run pip-audit (JSON + SARIF output) +3. Upload SARIF to GitHub Security tab +4. Upload JSON artifact (90-day retention) +5. Analyze and optionally fail on vulnerabilities + +**Features**: SARIF integration, configurable `fail_on_vulnerabilities` toggle, audit trail artifacts. + +#### Phase Snapshot (`phase-snapshot.yml`) + +Triggers on push to `phase-*` branches. + +**Job: validate** +- Run lint, typecheck, test, migrations +- Expose status outputs for downstream job + +**Job: create-snapshot** +- Extract phase number from branch name +- Generate `audit-data.json` + `requirements-frozen.txt` +- Create `SNAPSHOT-REPORT.md` +- Upload artifacts (365-day retention) +- Create annotated git tag: `phase-{N}-snapshot-{YYYYMMDD}-{sha}` + +**Purpose**: Creates audit snapshots at phase milestones. + +#### CD Release (`cd-release.yml`) + +Triggers on push to main. + +**Job: release-please** +- Parse conventional commits +- Create/update release PR +- Outputs: `release_created`, `tag_name`, `version` + +**Job: build-package** (runs if release created) +- Checkout at release tag +- Build Python package (`python -m build`) +- Upload to GitHub Release +- Store as workflow artifact + +**Supporting Files**: +| File | Purpose | +|------|---------| +| `release-please-config.json` | Package config (python type, version bumping) | +| `.release-please-manifest.json` | Version tracker (starts at 0.1.0) | + +**Conventional Commits → Versions**: +- `fix:` → patch (0.1.0 → 0.1.1) +- `feat:` → minor (0.1.0 → 0.2.0) +- `BREAKING CHANGE:` → major (0.1.0 → 1.0.0) + +#### Workflow Diagrams + +All workflow diagrams documented in `docs/github/diagrams/`: + +| Diagram | Description | +|---------|-------------| +| `cd-release-sequence.md` | Actor interactions during release | +| `cd-release-flow.md` | Release workflow job flow | +| `phase-snapshot-flow.md` | Phase-* branch snapshot flow | +| `schema-validation-flow.md` | Migration/drift check flow | +| `dependency-check-flow.md` | pip-audit scan flow | + +--- + ## Next Phase Preparation Phase 0 provides the foundation for: