From 7e00c30ca57bd9e1fd742c9c9953c7305dfe0ff2 Mon Sep 17 00:00:00 2001 From: "Gabe@w7dev" Date: Mon, 26 Jan 2026 08:39:39 +0000 Subject: [PATCH 1/2] docs: update phase-0 documentation with CI/CD infrastructure - Add section 9: CI/CD Infrastructure covering all GitHub Actions workflows - Document schema-validation, dependency-check, phase-snapshot, cd-release - Include workflow diagrams reference - Update PHASE-index with CI/CD deliverables and quick links - Condense code examples to stay under 600 lines Co-Authored-By: Claude Opus 4.5 --- docs/PHASE-index.md | 9 ++ docs/PHASE/0-INIT_PHASE.md | 259 ++++++++++++++++++------------------- 2 files changed, 135 insertions(+), 133 deletions(-) diff --git a/docs/PHASE-index.md b/docs/PHASE-index.md index ca710d2a..2bc0ae1b 100644 --- a/docs/PHASE-index.md +++ b/docs/PHASE-index.md @@ -42,6 +42,12 @@ This document indexes all implementation phases of the ForecastLabAI project. - `app/shared/` - Shared utilities (3 modules) - `app/main.py` - FastAPI application entry point - `alembic/` - Async migration setup +- `.github/workflows/` - CI/CD pipelines (5 workflows) + - `ci.yml` - Lint, typecheck, test, migration check + - `schema-validation.yml` - Migration drift detection + - `dependency-check.yml` - Weekly vulnerability scanning + - `phase-snapshot.yml` - Audit snapshots for phase-* branches + - `cd-release.yml` - Automated semantic versioning releases **Validation Results**: - Ruff: All checks passed @@ -102,6 +108,8 @@ Each phase document (`docs/PHASE/X-PHASE_NAME.md`) contains: - [Architecture Overview](./ARCHITECTURE.md) - [ADR Index](./ADR/ADR-INDEX.md) +- [GitHub Workflows Guide](./github/github-quickstart.md) +- [GitHub Workflow Diagrams](./github/diagrams/README.md) - [Logging Standard](./validation/logging-standard.md) - [MyPy Standard](./validation/mypy-standard.md) - [Pyright Standard](./validation/pyright-standard.md) @@ -115,3 +123,4 @@ Each phase document (`docs/PHASE/X-PHASE_NAME.md`) contains: | Date | Phase | Action | |------|-------|--------| | 2026-01-26 | 0 | Initial project foundation completed | +| 2026-01-26 | 0 | Added CI/CD infrastructure (5 GitHub Actions workflows) | diff --git a/docs/PHASE/0-INIT_PHASE.md b/docs/PHASE/0-INIT_PHASE.md index ed06b313..d3cc0cf7 100644 --- a/docs/PHASE/0-INIT_PHASE.md +++ b/docs/PHASE/0-INIT_PHASE.md @@ -190,155 +190,33 @@ def add_request_id( #### Database (`database.py`) -Async SQLAlchemy 2.0 setup: - -```python -class Base(DeclarativeBase): - pass - -def get_engine() -> AsyncEngine: - settings = get_settings() - return create_async_engine( - settings.database_url, - echo=settings.debug, - pool_pre_ping=True, - ) - -async def get_db() -> AsyncGenerator[AsyncSession, None]: - session_maker = get_session_maker() - async with session_maker() as session: - try: - yield session - await session.commit() - except Exception: - await session.rollback() - raise -``` - -**Features**: -- Async engine with connection pool -- Session dependency for FastAPI -- Auto-commit on success, rollback on failure -- Debug mode SQL echoing +Async SQLAlchemy 2.0 with session dependency: +- `Base` - DeclarativeBase for models +- `get_engine()` - Async engine with pool_pre_ping +- `get_db()` - FastAPI dependency with auto-commit/rollback #### Middleware (`middleware.py`) -Request ID correlation middleware: - -```python -class RequestIdMiddleware(BaseHTTPMiddleware): - async def dispatch( - self, - request: Request, - call_next: Callable[[Request], Awaitable[Response]], - ) -> Response: - request_id = request.headers.get("X-Request-ID") or str(uuid.uuid4()) - token = request_id_ctx.set(request_id) - - try: - logger.info("http.request_started", method=request.method, path=str(request.url.path)) - response = await call_next(request) - logger.info("http.request_completed", status_code=response.status_code) - response.headers["X-Request-ID"] = request_id - return response - finally: - request_id_ctx.reset(token) -``` - -**Features**: -- Generates UUID if no X-Request-ID header provided -- Preserves client-provided request IDs -- Injects request ID into all log entries -- Returns request ID in response header +Request ID correlation: generates UUID if no X-Request-ID header, preserves client-provided IDs, injects into all logs, returns in response header. #### Exceptions (`exceptions.py`) -Custom exception hierarchy with handlers: - -```python -class ForecastLabError(Exception): - def __init__( - self, - message: str, - code: str = "INTERNAL_ERROR", - status_code: int = 500, - details: dict[str, Any] | None = None, - ) -> None: - self.message = message - self.code = code - self.status_code = status_code - self.details = details or {} - -class NotFoundError(ForecastLabError): - # 404 errors - -class ValidationError(ForecastLabError): - # 422 errors - -class DatabaseError(ForecastLabError): - # 500 database errors -``` - -**Error Response Format**: -```json -{ - "error": { - "code": "NOT_FOUND", - "message": "Resource not found", - "details": {"resource_id": "123"}, - "request_id": "550e8400-e29b-41d4-a716-446655440000" - } -} -``` +Custom hierarchy: `ForecastLabError` base class with `NotFoundError` (404), `ValidationError` (422), `DatabaseError` (500). JSON error response includes code, message, details, request_id. #### Health Check (`health.py`) -Health and readiness endpoints: - | Endpoint | Purpose | Response | |----------|---------|----------| -| `GET /health` | Basic liveness check | `{"status": "ok"}` | -| `GET /health/ready` | Readiness with DB check | `{"status": "ok", "database": "connected"}` | +| `GET /health` | Liveness | `{"status": "ok"}` | +| `GET /health/ready` | Readiness + DB | `{"status": "ok", "database": "connected"}` | --- ### 4. Shared Utilities (`app/shared/`) -#### TimestampMixin (`models.py`) - -```python -class TimestampMixin: - created_at: Mapped[datetime] = mapped_column( - DateTime(timezone=True), - server_default=func.now(), - nullable=False, - ) - updated_at: Mapped[datetime] = mapped_column( - DateTime(timezone=True), - server_default=func.now(), - onupdate=func.now(), - nullable=False, - ) -``` - -#### Pagination (`schemas.py`) - -```python -class PaginationParams(BaseModel): - page: int = Field(1, ge=1) - page_size: int = Field(50, ge=1, le=1000) - - @property - def offset(self) -> int: - return (self.page - 1) * self.page_size - -class PaginatedResponse[T](BaseModel): - items: list[T] - total: int - page: int - page_size: int - pages: int -``` +- **TimestampMixin**: `created_at` and `updated_at` with server defaults +- **PaginationParams**: `page`, `page_size`, computed `offset` +- **PaginatedResponse[T]**: Generic paginated list with `items`, `total`, `pages` --- @@ -540,6 +418,121 @@ scripts/ --- +### 9. CI/CD Infrastructure (`.github/workflows/`) + +#### Core CI Pipeline (`ci.yml`) + +Runs on push/PR to main and dev branches: + +| Job | Purpose | Tools | +|-----|---------|-------| +| `lint` | Code quality | ruff check, ruff format | +| `typecheck` | Static analysis | mypy, pyright | +| `test` | Unit/integration tests | pytest with PostgreSQL | +| `migration-check` | Migration integrity | alembic upgrade head | + +**Key Features**: +- Concurrency groups to cancel stale runs +- PostgreSQL service container (pgvector:pg16) +- uv package manager with caching +- Python 3.12 environment + +#### Schema Validation (`schema-validation.yml`) + +Triggers on changes to `alembic/**` or `app/**/models.py`: + +```yaml +steps: + - Fresh DB migration test (alembic upgrade head) + - Migration chain integrity (single head enforcement) + - Schema drift detection (alembic check) + - Downgrade/upgrade cycle test + - Schema report generation +``` + +**Purpose**: Catches migration drift and schema issues before merge. + +#### Dependency Security Check (`dependency-check.yml`) + +Triggers weekly (Sunday 00:00 UTC) or manual dispatch: + +```yaml +steps: + - Export requirements from uv lock + - Run pip-audit (JSON + SARIF output) + - Upload SARIF to GitHub Security tab + - Upload JSON artifact (90-day retention) + - Analyze and optionally fail on vulnerabilities +``` + +**Features**: +- SARIF integration with GitHub Security tab +- Configurable `fail_on_vulnerabilities` toggle +- Audit trail artifacts + +#### Phase Snapshot (`phase-snapshot.yml`) + +Triggers on push to `phase-*` branches: + +```yaml +jobs: + validate: + - Lint, typecheck, test, migrations + - Expose status outputs + + create-snapshot: + - Extract phase number from branch + - Generate audit-data.json + requirements-frozen.txt + - Create SNAPSHOT-REPORT.md + - Upload artifacts (365-day retention) + - Create annotated git tag: phase-{N}-snapshot-{YYYYMMDD}-{sha} +``` + +**Purpose**: Creates audit snapshots at phase milestones. + +#### CD Release (`cd-release.yml`) + +Triggers on push to main: + +```yaml +jobs: + release-please: + - Parse conventional commits + - Create/update release PR + - Outputs: release_created, tag_name, version + + build-package: + - Checkout at release tag + - Build Python package (python -m build) + - Upload to GitHub Release + - Store as workflow artifact +``` + +**Supporting Files**: +| File | Purpose | +|------|---------| +| `release-please-config.json` | Package config (python type, version bumping) | +| `.release-please-manifest.json` | Version tracker (starts at 0.1.0) | + +**Conventional Commits → Versions**: +- `fix:` → patch (0.1.0 → 0.1.1) +- `feat:` → minor (0.1.0 → 0.2.0) +- `BREAKING CHANGE:` → major (0.1.0 → 1.0.0) + +#### Workflow Diagrams + +All workflow diagrams documented in `docs/github/diagrams/`: + +| Diagram | Description | +|---------|-------------| +| `cd-release-sequence.md` | Actor interactions during release | +| `cd-release-flow.md` | Release workflow job flow | +| `phase-snapshot-flow.md` | Phase-* branch snapshot flow | +| `schema-validation-flow.md` | Migration/drift check flow | +| `dependency-check-flow.md` | pip-audit scan flow | + +--- + ## Next Phase Preparation Phase 0 provides the foundation for: From 368afd3b48f7a9900182b513f4f405e19b986f5c Mon Sep 17 00:00:00 2001 From: "Gabe@w7dev" Date: Mon, 26 Jan 2026 08:44:09 +0000 Subject: [PATCH 2/2] fix(docs): address review comments on phase-0 documentation - Replace pseudo-YAML snippets with bullet lists for clarity - Restore error response JSON format example for correct usage - Restore pagination types code snippet for concrete context Co-Authored-By: Claude Opus 4.5 --- docs/PHASE/0-INIT_PHASE.md | 121 +++++++++++++++++++++---------------- 1 file changed, 68 insertions(+), 53 deletions(-) diff --git a/docs/PHASE/0-INIT_PHASE.md b/docs/PHASE/0-INIT_PHASE.md index d3cc0cf7..fa717d4b 100644 --- a/docs/PHASE/0-INIT_PHASE.md +++ b/docs/PHASE/0-INIT_PHASE.md @@ -201,7 +201,19 @@ Request ID correlation: generates UUID if no X-Request-ID header, preserves clie #### Exceptions (`exceptions.py`) -Custom hierarchy: `ForecastLabError` base class with `NotFoundError` (404), `ValidationError` (422), `DatabaseError` (500). JSON error response includes code, message, details, request_id. +Custom hierarchy: `ForecastLabError` base class with `NotFoundError` (404), `ValidationError` (422), `DatabaseError` (500). + +**Error Response Format**: +```json +{ + "error": { + "code": "NOT_FOUND", + "message": "Resource not found", + "details": {"resource_id": "123"}, + "request_id": "550e8400-e29b-41d4-a716-446655440000" + } +} +``` #### Health Check (`health.py`) @@ -215,8 +227,24 @@ Custom hierarchy: `ForecastLabError` base class with `NotFoundError` (404), `Val ### 4. Shared Utilities (`app/shared/`) - **TimestampMixin**: `created_at` and `updated_at` with server defaults -- **PaginationParams**: `page`, `page_size`, computed `offset` -- **PaginatedResponse[T]**: Generic paginated list with `items`, `total`, `pages` + +**Pagination Types** (`schemas.py`): +```python +class PaginationParams(BaseModel): + page: int = Field(1, ge=1) + page_size: int = Field(50, ge=1, le=1000) + + @property + def offset(self) -> int: + return (self.page - 1) * self.page_size + +class PaginatedResponse[T](BaseModel): + items: list[T] + total: int + page: int + page_size: int + pages: int +``` --- @@ -439,74 +467,61 @@ Runs on push/PR to main and dev branches: #### Schema Validation (`schema-validation.yml`) -Triggers on changes to `alembic/**` or `app/**/models.py`: +Triggers on changes to `alembic/**` or `app/**/models.py`. -```yaml -steps: - - Fresh DB migration test (alembic upgrade head) - - Migration chain integrity (single head enforcement) - - Schema drift detection (alembic check) - - Downgrade/upgrade cycle test - - Schema report generation -``` +**Steps:** +1. Fresh DB migration test (`alembic upgrade head`) +2. Migration chain integrity (single head enforcement) +3. Schema drift detection (`alembic check`) +4. Downgrade/upgrade cycle test +5. Schema report generation **Purpose**: Catches migration drift and schema issues before merge. #### Dependency Security Check (`dependency-check.yml`) -Triggers weekly (Sunday 00:00 UTC) or manual dispatch: +Triggers weekly (Sunday 00:00 UTC) or manual dispatch. -```yaml -steps: - - Export requirements from uv lock - - Run pip-audit (JSON + SARIF output) - - Upload SARIF to GitHub Security tab - - Upload JSON artifact (90-day retention) - - Analyze and optionally fail on vulnerabilities -``` +**Steps:** +1. Export requirements from uv lock +2. Run pip-audit (JSON + SARIF output) +3. Upload SARIF to GitHub Security tab +4. Upload JSON artifact (90-day retention) +5. Analyze and optionally fail on vulnerabilities -**Features**: -- SARIF integration with GitHub Security tab -- Configurable `fail_on_vulnerabilities` toggle -- Audit trail artifacts +**Features**: SARIF integration, configurable `fail_on_vulnerabilities` toggle, audit trail artifacts. #### Phase Snapshot (`phase-snapshot.yml`) -Triggers on push to `phase-*` branches: +Triggers on push to `phase-*` branches. -```yaml -jobs: - validate: - - Lint, typecheck, test, migrations - - Expose status outputs - - create-snapshot: - - Extract phase number from branch - - Generate audit-data.json + requirements-frozen.txt - - Create SNAPSHOT-REPORT.md - - Upload artifacts (365-day retention) - - Create annotated git tag: phase-{N}-snapshot-{YYYYMMDD}-{sha} -``` +**Job: validate** +- Run lint, typecheck, test, migrations +- Expose status outputs for downstream job + +**Job: create-snapshot** +- Extract phase number from branch name +- Generate `audit-data.json` + `requirements-frozen.txt` +- Create `SNAPSHOT-REPORT.md` +- Upload artifacts (365-day retention) +- Create annotated git tag: `phase-{N}-snapshot-{YYYYMMDD}-{sha}` **Purpose**: Creates audit snapshots at phase milestones. #### CD Release (`cd-release.yml`) -Triggers on push to main: +Triggers on push to main. -```yaml -jobs: - release-please: - - Parse conventional commits - - Create/update release PR - - Outputs: release_created, tag_name, version - - build-package: - - Checkout at release tag - - Build Python package (python -m build) - - Upload to GitHub Release - - Store as workflow artifact -``` +**Job: release-please** +- Parse conventional commits +- Create/update release PR +- Outputs: `release_created`, `tag_name`, `version` + +**Job: build-package** (runs if release created) +- Checkout at release tag +- Build Python package (`python -m build`) +- Upload to GitHub Release +- Store as workflow artifact **Supporting Files**: | File | Purpose |