feat(release): cut v0.2.15 for explainability, scenario simulation & ops control center (#232)

.env.example

@@ -97,7 +97,7 @@ AGENT_SESSION_TTL_MINUTES=120
 AGENT_MAX_SESSIONS_PER_USER=5
 
 # Human-in-the-loop actions (JSON array format required for safe parsing)
-AGENT_REQUIRE_APPROVAL=["create_alias","archive_run"]
+AGENT_REQUIRE_APPROVAL=["create_alias","archive_run","save_scenario"]
 AGENT_APPROVAL_TIMEOUT_MINUTES=60
 
 # Streaming

.gitignore

@@ -27,6 +27,7 @@ infra/terraform.tfvars
 .terraform
 .terraform*
 .langgraph_api
+.mcp.json
 
 # Virtual environments
 .venv

CLAUDE.md

@@ -21,8 +21,8 @@ Claude pulls these in on demand — load only what the task touches.
 - Pipeline contract (CI/CD):     @docs/_base/PIPELINE_CONTRACT.md
 
 > Project rules are enforced via `.claude/rules/` (commit-format, branch-naming,
-> security-patterns, product-vision, test-requirements, ui-design, versioning,
-> output-formatting). Read those first — they are authoritative on detail.
+> security-patterns, product-vision, test-requirements, ui-design, shadcn-ui,
+> versioning, output-formatting). Read those first — they are authoritative on detail.
 
 ## Safety
 

PRPs/ai_docs/exogenous-regressor-forecasting.md

@@ -0,0 +1,164 @@
+# Exogenous-Regressor Forecasting & Leakage-Safe Future Feature Frames
+
+> Curated reference for **PRP-27 (Scenario Simulation — Full Version)**. ForecastLabAI's
+> baseline forecasters (`naive`, `seasonal_naive`, `moving_average`) ignore the exogenous
+> `X` argument (every `fit`/`predict` carries `# noqa: ARG002`). The Full Version needs a
+> forecaster that *consumes* `X` so a scenario assumption can be expressed as a real
+> regressor change instead of a post-forecast multiplier. This doc condenses the parts of
+> the LightGBM / scikit-learn / pandas docs that matter for that, plus the leakage rule.
+
+---
+
+## 1. The exogenous-regressor model contract (what to build)
+
+A "regression-on-features" forecaster predicts demand from a **feature row per future
+day**, not from the historical target series. The flow:
+
+```
+TRAIN:   y, X_hist  ─fit─►  estimator               (X_hist built by featuresets, cutoff-safe)
+PREDICT: X_future   ─predict─►  ŷ_future             (X_future = the future feature frame)
+```
+
+- `X_hist` is a 2-D array `[n_samples, n_features]` — the columns featuresets already
+  produces (`lag_*`, `rolling_*`, calendar, `price_lag_*`, `promo_*`, lifecycle).
+- `X_future` is the **same columns** for the horizon days. This is the *future feature
+  frame* — the central new artifact of PRP-27.
+- The estimator is a gradient-boosted tree regressor (`LGBMRegressor`) — or, to avoid a
+  new dependency, scikit-learn's `HistGradientBoostingRegressor` (already in the
+  `scikit-learn` dep). **Prefer the scikit-learn option** — see §5.
+
+### scikit-learn `HistGradientBoostingRegressor` (no new dependency)
+
+```python
+from sklearn.ensemble import HistGradientBoostingRegressor
+
+est = HistGradientBoostingRegressor(
+    max_iter=200, learning_rate=0.05, max_depth=6, random_state=42,
+)
+est.fit(X_hist, y)            # X_hist: ndarray [n, k]; y: ndarray [n]
+y_future = est.predict(X_future)   # X_future: ndarray [horizon, k]
+```
+
+- Histogram-based, fast, handles `NaN` natively (important — lag features have `NaN`
+  at series start). Deterministic with a fixed `random_state`.
+- Docs: https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.HistGradientBoostingRegressor.html
+
+### LightGBM `LGBMRegressor` (only if a new dependency is approved)
+
+```python
+from lightgbm import LGBMRegressor
+est = LGBMRegressor(n_estimators=200, learning_rate=0.05, max_depth=6,
+                    random_state=42, n_jobs=1, verbose=-1)
+est.fit(X_hist, y)
+y_future = est.predict(X_future)
+```
+
+- API: https://lightgbm.readthedocs.io/en/stable/pythonapi/lightgbm.LGBMRegressor.html
+- Set `n_jobs=1` + `random_state` for reproducibility; `verbose=-1` to silence.
+- `LightGBMModelConfig` already exists in `forecasting/schemas.py` and
+  `forecast_enable_lightgbm` already exists in config — but **LightGBM is NOT in
+  `pyproject.toml`** and `model_factory` raises `NotImplementedError`. Adding it is a
+  `pyproject.toml` change + a stop-and-ask gate (see PRP-27 § Vision Tensions).
+
+---
+
+## 2. The leakage rule for FUTURE feature frames (the load-bearing part)
+
+`app/features/featuresets/service.py` builds **historical** features and is time-safe by
+construction: it filters to `cutoff_date` *before* any compute, lags via `shift(positive)`,
+rolls via `shift(1).rolling(...)`, all `groupby` entity-aware. `test_leakage.py` is its spec.
+
+A **future** feature frame is different and dangerous: for horizon day `D` you must
+produce the SAME feature columns, but `D` has **no observed target**. The rule:
+
+> **A future feature row for day `D` may only use information available at the forecast
+> origin `T` (the last training day) — never an observed value at `D` or later.**
+
+Concretely, for a horizon `T+1 … T+H`:
+
+| Feature family | How to populate the future frame | Leakage trap to avoid |
+|----------------|----------------------------------|------------------------|
+| `lag_k` (k ≥ horizon) | Real observed `y[T+1-k]` — available at `T`. | — |
+| `lag_k` (k < horizon) | **Recursive**: `lag_k` at `T+j` = the model's own prediction `ŷ[T+j-k]`. NEVER a real future `y`. | Using a real `y[T+j-k]` (does not exist) or 0. |
+| `rolling_*` | Built from the same `shift(1)`-then-roll over the *extended* (history + predicted) series. | Rolling over un-shifted future values. |
+| calendar (`dow`, `month`, `is_weekend`, …) | Pure function of the date `D` — always safe, compute directly. | — |
+| `price_lag_*`, `promo_*` | Driven by the **scenario assumptions** — the planner is *positing* a future price/promo. This is the intended what-if input, not leakage. | Reading real future `price_history` rows. |
+| `is_holiday` | From the scenario's holiday assumption OR the `calendar` table (a `calendar` row is a timeless attribute, like `launch_date`). | — |
+| lifecycle (`days_since_launch`) | Pure function of `D - product.launch_date` — safe. | — |
+
+**Recursive (iterative) forecasting** is the standard technique for multi-step horizons
+when lags shorter than the horizon exist: predict `T+1`, append `ŷ[T+1]` to the working
+series, recompute lags, predict `T+2`, and so on. Pandas time-series guide:
+https://pandas.pydata.org/docs/user_guide/timeseries.html
+
+**Simplification that sidesteps recursion entirely:** if the future feature frame uses
+ONLY lags `k ≥ horizon`, calendar features, and assumption-driven exogenous columns, then
+every feature value is knowable at `T` with no recursion. PRP-27 recommends this
+"long-lag + exogenous + calendar" feature set for the MVP of the Full Version — it keeps
+the leakage proof tractable (`test_leakage.py` can assert it directly) and is one-pass
+implementable. Recursion is a documented Phase-2 extension.
+
+---
+
+## 3. Why this is leakage-critical for a planner
+
+The MVP (PRP-26) is *immune* to leakage because it never builds a future feature frame —
+it multiplies the baseline forecast by a deterministic factor. The Full Version
+*introduces* the future feature frame, so it introduces the leakage surface the MVP did
+not have. PRP-27 therefore ships a NEW load-bearing test
+`app/features/scenarios/tests/test_leakage.py` extension (or a sibling
+`test_future_frame_leakage.py`) that asserts the future-frame generator never reads an
+observed target at or after the forecast origin. This mirrors
+`app/features/featuresets/tests/test_leakage.py` — never weaken it to make a feature pass.
+
+---
+
+## 4. Multi-scenario comparison (UX + math)
+
+Comparing N scenarios against one baseline is an aggregation over N `ScenarioComparison`
+objects:
+
+- Each scenario contributes one `(units_delta, revenue_delta, coverage_verdict)` triple.
+- The comparison view ranks scenarios by a chosen metric (revenue delta default) and
+  renders all series on one chart (baseline + one line per scenario).
+- Recharts renders M+1 `<Line>` series from one merged row array keyed by date —
+  `frontend/src/components/charts/time-series-chart.tsx` currently wraps a 2-series case;
+  a multi-series variant passes a `series: {key,label,color}[]` prop. Recharts LineChart:
+  https://recharts.org/en-US/api/LineChart
+- TanStack Query: the comparison page issues one query per saved scenario id (or one
+  batch endpoint). Mutations vs queries pattern:
+  https://tanstack.com/query/latest/docs/framework/react/guides/mutations
+
+---
+
+## 5. Recommendation for PRP-27 (de-risking)
+
+1. **Prefer `HistGradientBoostingRegressor`** over LightGBM — it is already a transitive
+   dependency via `scikit-learn`, so no `pyproject.toml` change and no stop-and-ask gate.
+   It is deterministic, NaN-tolerant, and fast enough for single-series horizons.
+2. **Use the long-lag + calendar + exogenous feature set** so the future frame needs no
+   recursion — the leakage proof stays simple and the PRP stays one-pass implementable.
+3. **Keep `method` forward-compatible** — the MVP locked `method="heuristic"` behind a
+   CHECK constraint. The Full Version adds `method="model_exogenous"`; the migration must
+   widen the CHECK to `IN ('heuristic','model_exogenous')`.
+4. **Never replace the heuristic path** — it stays as the fallback when a baseline model
+   does not support exogenous features. A scenario result always declares which `method`
+   produced it, and the heuristic disclaimer stays on heuristic results.
+
+---
+
+## Source URLs (with the sections that matter)
+
+- scikit-learn `HistGradientBoostingRegressor` — fit/predict, NaN handling, `random_state`:
+  https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.HistGradientBoostingRegressor.html
+- scikit-learn `TimeSeriesSplit` — for any backtest of the exogenous model:
+  https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.TimeSeriesSplit.html
+- LightGBM `LGBMRegressor` Python API — only if the dependency is approved:
+  https://lightgbm.readthedocs.io/en/stable/pythonapi/lightgbm.LGBMRegressor.html
+- pandas time-series user guide — date ranges, shifting, rolling for the future frame:
+  https://pandas.pydata.org/docs/user_guide/timeseries.html
+- Recharts LineChart — multi-series scenario comparison chart:
+  https://recharts.org/en-US/api/LineChart
+- NIST AI Risk Management Framework — transparency controls for model-driven revenue
+  claims (the `disclaimer` / `method` labelling requirement):
+  https://www.nist.gov/itl/ai-risk-management-framework

README.md

@@ -11,6 +11,7 @@ Portfolio-grade end-to-end retail demand forecasting system.
 - **Dashboard**: React 19 + Vite + Tailwind CSS 4 + shadcn/ui for data exploration and model management
 - **Explorer**: Click-through detail pages for stores, products, model runs, and jobs; run-vs-run comparison and SHA-256 artifact integrity verification; server-side sortable, CSV-exportable tables with column-visibility toggles and URL-shareable filter/sort/page state across every Explorer page; date-scoped KPIs, revenue bar/line charts, and cross-filtering on the Sales page
 - **Demand Planner**: `/visualize/demand` — every completed forecast rolled into a multi-SKU table (tomorrow / next-week / next-month demand + inventory requirement), with a lead-time selector and a single-SKU drill-in; the Forecast and Backtest pages run jobs in-page, export CSV, toggle a prediction-interval band, and cross-link to runs/jobs
+- **What-If Planner**: `/visualize/planner` — take an existing forecast, apply price / promotion / holiday / inventory / lifecycle assumptions, and see the baseline-vs-scenario demand and revenue impact; a regression baseline genuinely re-forecasts through the assumptions (`method="model_exogenous"`), any other baseline applies a clearly-labelled deterministic heuristic; save, tag, reload, clone, and delete named scenario plans, and rank 2-5 saved plans side by side in a multi-scenario comparison. The experiment chat agent can also propose a scenario and — behind the human-in-the-loop approval gate — save it for you
 - **RAG Knowledge Base**: Postgres pgvector embeddings + evidence-grounded answers with citations
 - **Agentic Layer**: PydanticAI agents for autonomous experimentation and evidence-grounded Q&A with human-in-the-loop approval
 - **Data Seeder (The Forge)**: Reproducible synthetic data generator with realistic time-series patterns, scenario presets, and retail effects

alembic/env.py

@@ -15,9 +15,11 @@
 from app.features.agents import models as agents_models  # noqa: F401
 from app.features.config import models as config_models  # noqa: F401
 from app.features.data_platform import models as data_platform_models  # noqa: F401
+from app.features.explainability import models as explainability_models  # noqa: F401
 from app.features.jobs import models as jobs_models  # noqa: F401
 from app.features.rag import models as rag_models  # noqa: F401
 from app.features.registry import models as registry_models  # noqa: F401
+from app.features.scenarios import models as scenarios_models  # noqa: F401
 
 # Alembic Config object
 config = context.config

alembic/versions/43e35957a248_create_scenario_plan_table.py

@@ -0,0 +1,97 @@
+"""create scenario plan table
+
+Revision ID: 43e35957a248
+Revises: 378c112e4b32
+Create Date: 2026-05-19 07:34:30.545495
+
+"""
+from typing import Sequence, Union
+
+from alembic import op
+import sqlalchemy as sa
+from sqlalchemy.dialects import postgresql
+
+# revision identifiers, used by Alembic.
+revision: str = '43e35957a248'
+down_revision: Union[str, None] = '378c112e4b32'
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    """Apply migration — create the scenario_plan table."""
+    op.create_table(
+        'scenario_plan',
+        sa.Column('id', sa.Integer(), nullable=False),
+        sa.Column('scenario_id', sa.String(length=32), nullable=False),
+        sa.Column('name', sa.String(length=200), nullable=False),
+        sa.Column('store_id', sa.Integer(), nullable=False),
+        sa.Column('product_id', sa.Integer(), nullable=False),
+        sa.Column('run_id', sa.String(length=32), nullable=False),
+        sa.Column('horizon', sa.Integer(), nullable=False),
+        sa.Column('assumptions', postgresql.JSONB(astext_type=sa.Text()), nullable=False),
+        sa.Column('comparison', postgresql.JSONB(astext_type=sa.Text()), nullable=False),
+        sa.Column('method', sa.String(length=20), nullable=False),
+        sa.Column(
+            'created_at',
+            sa.DateTime(timezone=True),
+            server_default=sa.text('now()'),
+            nullable=False,
+        ),
+        sa.Column(
+            'updated_at',
+            sa.DateTime(timezone=True),
+            server_default=sa.text('now()'),
+            nullable=False,
+        ),
+        sa.CheckConstraint("method IN ('heuristic')", name='ck_scenario_plan_method'),
+        sa.PrimaryKeyConstraint('id'),
+    )
+    op.create_index(
+        op.f('ix_scenario_plan_scenario_id'), 'scenario_plan', ['scenario_id'], unique=True
+    )
+    op.create_index(
+        op.f('ix_scenario_plan_store_id'), 'scenario_plan', ['store_id'], unique=False
+    )
+    op.create_index(
+        op.f('ix_scenario_plan_product_id'), 'scenario_plan', ['product_id'], unique=False
+    )
+    op.create_index(
+        op.f('ix_scenario_plan_run_id'), 'scenario_plan', ['run_id'], unique=False
+    )
+    op.create_index(
+        'ix_scenario_plan_assumptions_gin',
+        'scenario_plan',
+        ['assumptions'],
+        unique=False,
+        postgresql_using='gin',
+    )
+    op.create_index(
+        'ix_scenario_plan_comparison_gin',
+        'scenario_plan',
+        ['comparison'],
+        unique=False,
+        postgresql_using='gin',
+    )
+    op.create_index(
+        'ix_scenario_plan_store_product',
+        'scenario_plan',
+        ['store_id', 'product_id'],
+        unique=False,
+    )
+
+
+def downgrade() -> None:
+    """Revert migration — drop the scenario_plan table."""
+    op.drop_index('ix_scenario_plan_store_product', table_name='scenario_plan')
+    op.drop_index(
+        'ix_scenario_plan_comparison_gin', table_name='scenario_plan', postgresql_using='gin'
+    )
+    op.drop_index(
+        'ix_scenario_plan_assumptions_gin', table_name='scenario_plan', postgresql_using='gin'
+    )
+    op.drop_index(op.f('ix_scenario_plan_run_id'), table_name='scenario_plan')
+    op.drop_index(op.f('ix_scenario_plan_product_id'), table_name='scenario_plan')
+    op.drop_index(op.f('ix_scenario_plan_store_id'), table_name='scenario_plan')
+    op.drop_index(op.f('ix_scenario_plan_scenario_id'), table_name='scenario_plan')
+    op.drop_table('scenario_plan')

alembic/versions/7e8f9748581e_add_scenario_provenance_columns.py

@@ -0,0 +1,73 @@
+"""add scenario provenance columns
+
+Revision ID: 7e8f9748581e
+Revises: bb8c4587ef1d
+Create Date: 2026-05-19 10:47:09.829097
+
+PRP-27 Phase D — adds provenance + approval-audit columns to ``scenario_plan``
+so an agent-proposed plan records who/what created it and the human approval
+decision that released it. ``source`` server-defaults to ``'user'`` so every
+pre-existing row stays valid. Forward-only.
+"""
+from typing import Sequence, Union
+
+from alembic import op
+import sqlalchemy as sa
+
+# revision identifiers, used by Alembic.
+revision: str = '7e8f9748581e'
+down_revision: Union[str, None] = 'bb8c4587ef1d'
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    """Add the source + approval-audit columns, their CHECKs and an index."""
+    op.add_column(
+        'scenario_plan',
+        sa.Column(
+            'source',
+            sa.String(length=16),
+            nullable=False,
+            server_default=sa.text("'user'"),
+        ),
+    )
+    op.add_column(
+        'scenario_plan',
+        sa.Column('agent_session_id', sa.String(length=32), nullable=True),
+    )
+    op.add_column(
+        'scenario_plan',
+        sa.Column('approved_by', sa.String(length=120), nullable=True),
+    )
+    op.add_column(
+        'scenario_plan',
+        sa.Column('approved_at', sa.DateTime(timezone=True), nullable=True),
+    )
+    op.add_column(
+        'scenario_plan',
+        sa.Column('approval_decision', sa.String(length=16), nullable=True),
+    )
+    op.create_check_constraint(
+        'ck_scenario_plan_source',
+        'scenario_plan',
+        "source IN ('user', 'agent')",
+    )
+    op.create_check_constraint(
+        'ck_scenario_plan_approval_decision',
+        'scenario_plan',
+        "approval_decision IS NULL OR approval_decision IN ('approved', 'rejected')",
+    )
+    op.create_index('ix_scenario_plan_source', 'scenario_plan', ['source'], unique=False)
+
+
+def downgrade() -> None:
+    """Drop the index, the two CHECKs and the provenance columns."""
+    op.drop_index('ix_scenario_plan_source', table_name='scenario_plan')
+    op.drop_constraint('ck_scenario_plan_approval_decision', 'scenario_plan', type_='check')
+    op.drop_constraint('ck_scenario_plan_source', 'scenario_plan', type_='check')
+    op.drop_column('scenario_plan', 'approval_decision')
+    op.drop_column('scenario_plan', 'approved_at')
+    op.drop_column('scenario_plan', 'approved_by')
+    op.drop_column('scenario_plan', 'agent_session_id')
+    op.drop_column('scenario_plan', 'source')