Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .release-please-manifest.json
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
{
".": "0.2.12"
".": "0.2.13"
}
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# Changelog

## [0.2.13](https://github.com/w7-mgfcode/ForecastLabAI/compare/v0.2.12...v0.2.13) (2026-05-18)


### Features

* cut v0.2.13 — explorer interactivity, knowledge & guide pages ([#191](https://github.com/w7-mgfcode/ForecastLabAI/issues/191)) ([#192](https://github.com/w7-mgfcode/ForecastLabAI/issues/192)) ([ae37ca5](https://github.com/w7-mgfcode/ForecastLabAI/commit/ae37ca521eb9510c135def4a1e3730e137fb014b))

## [0.2.12](https://github.com/w7-mgfcode/ForecastLabAI/compare/v0.2.11...v0.2.12) (2026-05-18)


Expand Down
952 changes: 952 additions & 0 deletions PRPs/PRP-19-knowledge-and-agent-guide-pages.md

Large diffs are not rendered by default.

1,002 changes: 1,002 additions & 0 deletions PRPs/PRP-20-explorer-interactivity.md

Large diffs are not rendered by default.

1,022 changes: 1,022 additions & 0 deletions PRPs/PRP-21-explorer-runs-jobs-interactivity.md

Large diffs are not rendered by default.

5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,13 @@ Portfolio-grade end-to-end retail demand forecasting system.
- **Serving Layer**: Typed FastAPI endpoints (Pydantic v2 validation)
- **Model Registry**: Run configs, metrics, artifacts, and data windows for reproducibility
- **Dashboard**: React 19 + Vite + Tailwind CSS 4 + shadcn/ui for data exploration and model management
- **Explorer**: Click-through detail pages for stores, products, model runs, and jobs; run-vs-run comparison and SHA-256 artifact integrity verification; server-side sortable, CSV-exportable tables with column-visibility toggles and URL-shareable filter/sort/page state across every Explorer page; date-scoped KPIs, revenue bar/line charts, and cross-filtering on the Sales page
- **RAG Knowledge Base**: Postgres pgvector embeddings + evidence-grounded answers with citations
- **Agentic Layer**: PydanticAI agents for autonomous experimentation and evidence-grounded Q&A with human-in-the-loop approval
- **Data Seeder (The Forge)**: Reproducible synthetic data generator with realistic time-series patterns, scenario presets, and retail effects
- **AI Models Console**: `/admin` → AI Models tab — swap the agent LLM (incl. fully-local Ollama), the RAG embedding model, and provider API keys at runtime; changes apply live with no restart
- **Knowledge Page**: `/knowledge` — browse the indexed RAG corpus, run a live semantic search, and see the live system state (seeded data, model runs, deployment aliases) the agents draw on
- **Agent Guide**: `/guide` — in-product reference for the two chat agents — their tools, the human-in-the-loop approval gate, live session limits, and copy-paste example prompts

## Quick Start

Expand Down Expand Up @@ -449,11 +452,13 @@ curl "http://localhost:8123/dimensions/products?search=Cola&category=Beverage"
- 1-indexed pagination (page=1 is first page)
- Case-insensitive search in code/sku and name fields
- Filter by region, store_type, category, or brand
- Optional `sort_by` / `sort_order` on the store and product lists (allow-listed columns; unknown values fall back to the default order)

### Analytics

- `GET /analytics/kpis` - Compute aggregated KPIs for a date range
- `GET /analytics/drilldowns` - Drill into data by dimension (store, product, category, region, date)
- `GET /analytics/timeseries` - Period-bucketed sales series (day/week/month/quarter) for revenue-over-time charts

**Example KPI Request:**
```bash
Expand Down
51 changes: 51 additions & 0 deletions app/features/agents/agents/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
import structlog
from pydantic_ai import ModelRetry
from pydantic_ai.models import Model
from pydantic_ai.models.fallback import FallbackModel
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.ollama import OllamaProvider

Expand Down Expand Up @@ -120,6 +121,56 @@ def get_fallback_model() -> str:
return settings.agent_fallback_model


def build_agent_model_with_fallback() -> Model | str:
"""Build the PydanticAI ``model`` argument, wrapping primary + fallback.

When the primary model raises a provider error — HTTP 5xx, rate limit,
timeout, i.e. any ``pydantic_ai.exceptions.ModelAPIError`` — PydanticAI's
:class:`FallbackModel` transparently retries the request against
``agent_fallback_model``. This keeps an agent run alive through a transient
provider outage (e.g. a Gemini ``503 UNAVAILABLE``) instead of surfacing a
hard error. ``FallbackModel``'s default ``fallback_on=(ModelAPIError,)``
already covers that case.

The primary model is returned alone (no fallback wrapper) when:

- no fallback is configured, or it equals the primary identifier; or
- the fallback provider has no API key — wrapping it would only move the
failure, so the agent runs primary-only and logs a warning.

Returns:
A :class:`FallbackModel` (primary then fallback) when a usable fallback
is configured, otherwise the primary model argument from
:func:`build_agent_model`.

Raises:
ValueError: If the primary provider's API key is not configured
(fail-fast — an agent with no usable primary cannot run).
"""
primary_id = get_model_identifier()
validate_api_key_for_model(primary_id) # fail-fast on the primary
primary = build_agent_model(primary_id)

fallback_id = get_fallback_model()
if not fallback_id or fallback_id == primary_id:
return primary

try:
validate_api_key_for_model(fallback_id)
except ValueError:
logger.warning(
"agents.fallback_disabled",
reason="missing_api_key",
primary=primary_id,
fallback=fallback_id,
)
return primary

fallback = build_agent_model(fallback_id)
logger.info("agents.fallback_enabled", primary=primary_id, fallback=fallback_id)
return FallbackModel(primary, fallback)


def get_agent_retries() -> int:
"""Get the configured retry budget for agent tool calls and output validation.

Expand Down
10 changes: 4 additions & 6 deletions app/features/agents/agents/experiment.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,11 @@
SAFETY_INSTRUCTIONS,
SYSTEM_PROMPT_HEADER,
TOOL_USAGE_INSTRUCTIONS,
build_agent_model,
build_agent_model_with_fallback,
get_agent_retries,
get_model_identifier,
get_model_settings,
recoverable,
requires_approval,
validate_api_key_for_model,
)
from app.features.agents.deps import AgentDeps
from app.features.agents.schemas import ExperimentReport
Expand Down Expand Up @@ -83,9 +81,9 @@ def create_experiment_agent() -> Agent[AgentDeps, ExperimentReport]:
Returns:
Configured Agent instance with tools registered.
"""
identifier = get_model_identifier()
validate_api_key_for_model(identifier) # Fail-fast validation
model = build_agent_model(identifier) # str for cloud, Model object for ollama
# Primary model, wrapped in a FallbackModel so a transient provider error
# (HTTP 5xx, rate limit) on the primary transparently retries the fallback.
model = build_agent_model_with_fallback()

retries = get_agent_retries()
agent: Agent[AgentDeps, ExperimentReport] = Agent(
Expand Down
10 changes: 4 additions & 6 deletions app/features/agents/agents/rag_assistant.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,10 @@
from app.features.agents.agents.base import (
SAFETY_INSTRUCTIONS,
SYSTEM_PROMPT_HEADER,
build_agent_model,
build_agent_model_with_fallback,
get_agent_retries,
get_model_identifier,
get_model_settings,
recoverable,
validate_api_key_for_model,
)
from app.features.agents.deps import AgentDeps
from app.features.agents.schemas import RAGAnswer
Expand Down Expand Up @@ -78,9 +76,9 @@ def create_rag_assistant_agent() -> Agent[AgentDeps, RAGAnswer]:
Returns:
Configured Agent instance with tools registered.
"""
identifier = get_model_identifier()
validate_api_key_for_model(identifier) # Fail-fast validation
model = build_agent_model(identifier) # str for cloud, Model object for ollama
# Primary model, wrapped in a FallbackModel so a transient provider error
# (HTTP 5xx, rate limit) on the primary transparently retries the fallback.
model = build_agent_model_with_fallback()

retries = get_agent_retries()
agent: Agent[AgentDeps, RAGAnswer] = Agent(
Expand Down
62 changes: 62 additions & 0 deletions app/features/agents/tests/test_base.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,15 @@
import pytest
from pydantic_ai import ModelRetry
from pydantic_ai.messages import ModelMessage, ModelResponse, TextPart
from pydantic_ai.models.fallback import FallbackModel
from pydantic_ai.models.function import AgentInfo, FunctionModel
from pydantic_ai.models.openai import OpenAIChatModel

from app.core.config import get_settings
from app.features.agents.agents.base import (
TOOL_USAGE_INSTRUCTIONS,
build_agent_model,
build_agent_model_with_fallback,
get_agent_retries,
recoverable,
validate_api_key_for_model,
Expand Down Expand Up @@ -62,6 +64,66 @@ def test_validate_api_key_for_model_ollama_skips_key_check():
validate_api_key_for_model("ollama:llama3.1")


def test_build_agent_model_with_fallback_wraps_primary_and_fallback():
"""A distinct, key-backed fallback yields a FallbackModel wired primary-then-fallback.

Asserts the *order* via the public ``FallbackModel.models`` list — ``models[0]``
must be the primary (``agent_default_model``) and ``models[1]`` the fallback
(``agent_fallback_model``) — so a swap or misconfiguration is caught, not just
the wrapper type.
"""
settings = get_settings()
settings.agent_default_model = "anthropic:claude-sonnet-4-5"
settings.agent_fallback_model = "openai:gpt-4o"
settings.anthropic_api_key = "test-anthropic-key"
settings.openai_api_key = "test-openai-key"

model = build_agent_model_with_fallback()

assert isinstance(model, FallbackModel)
# Each member model exposes its provider via `.system` and name via
# `.model_name`; recombine to the `provider:model` identifier we configured.
wired = [f"{m.system}:{m.model_name}" for m in model.models]
assert wired == [settings.agent_default_model, settings.agent_fallback_model]


def test_build_agent_model_with_fallback_raises_when_primary_api_key_missing():
"""Fail fast: a non-Ollama primary with no API key raises before any wrapping."""
settings = get_settings()
settings.agent_default_model = "anthropic:claude-sonnet-4-5"
settings.agent_fallback_model = "openai:gpt-4o"
settings.anthropic_api_key = ""
settings.openai_api_key = "test-openai-key"

with pytest.raises(ValueError, match="Anthropic API key not configured"):
build_agent_model_with_fallback()


def test_build_agent_model_with_fallback_primary_only_when_fallback_key_missing():
"""With no API key for the fallback provider, the primary is returned alone."""
settings = get_settings()
settings.agent_default_model = "anthropic:claude-sonnet-4-5"
settings.agent_fallback_model = "openai:gpt-4o"
settings.anthropic_api_key = "test-anthropic-key"
settings.openai_api_key = ""

model = build_agent_model_with_fallback()

assert model == "anthropic:claude-sonnet-4-5"


def test_build_agent_model_with_fallback_primary_only_when_fallback_equals_primary():
"""A fallback identical to the primary adds no resilience — primary returned alone."""
settings = get_settings()
settings.agent_default_model = "anthropic:claude-sonnet-4-5"
settings.agent_fallback_model = "anthropic:claude-sonnet-4-5"
settings.anthropic_api_key = "test-anthropic-key"

model = build_agent_model_with_fallback()

assert model == "anthropic:claude-sonnet-4-5"


def test_prompts_only_reference_registered_tool_names() -> None:
"""Every `tool_*` name in the agent prompts must be an actually-registered tool.

Expand Down
95 changes: 95 additions & 0 deletions app/features/analytics/routes.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@
DrilldownDimension,
DrilldownResponse,
KPIResponse,
TimeGranularity,
TimeSeriesResponse,
)
from app.features.analytics.service import AnalyticsService

Expand Down Expand Up @@ -247,3 +249,96 @@ async def get_drilldowns(
product_id=product_id,
max_items=max_items,
)


# =============================================================================
# Time Series Endpoints
# =============================================================================


@router.get(
"/timeseries",
response_model=TimeSeriesResponse,
summary="Compute a period-bucketed sales time series",
description="""
Aggregate sales into a time series bucketed by day, week, month, or quarter.

**Purpose**: Drive revenue-over-time charts. Unlike `/drilldowns?dimension=date`,
this endpoint orders points by period (not revenue), supports week/month/quarter
bucketing, and is not capped at 100 items.

**Metrics per period**: same `KPIMetrics` shape as `/analytics/kpis` —
`total_revenue`, `total_units`, `total_transactions`, `avg_unit_price`,
`avg_basket_value`.

**Filtering Options**:
- `store_id`: scope the series to a single store
- `product_id`: scope the series to a single product
- `category`: scope the series to a product category (exact match)

**Date Range**:
- Both `start_date` and `end_date` are inclusive
- Maximum range: 730 days (2 years)

**Example Use Cases**:
1. Daily revenue trend: `GET /analytics/timeseries?start_date=2024-01-01&end_date=2024-03-31&granularity=day`
2. Weekly trend for a store: `GET /analytics/timeseries?store_id=5&start_date=2024-01-01&end_date=2024-12-31&granularity=week`
""",
)
async def get_timeseries(
start_date: date = Query(
...,
description="Start of analysis period (inclusive). Format: YYYY-MM-DD.",
),
end_date: date = Query(
...,
description="End of analysis period (inclusive). Format: YYYY-MM-DD.",
),
granularity: TimeGranularity = Query(
TimeGranularity.DAY,
description="Bucket size: day, week, month, or quarter.",
),
store_id: int | None = Query(
None,
description="Filter by store ID. Use GET /dimensions/stores to find valid IDs.",
),
product_id: int | None = Query(
None,
description="Filter by product ID. Use GET /dimensions/products to find valid IDs.",
),
category: str | None = Query(
None,
description="Filter by product category name (exact match).",
),
db: AsyncSession = Depends(get_db),
) -> TimeSeriesResponse:
"""Compute a period-bucketed sales time series with optional filters.

Args:
start_date: Start of analysis period (inclusive).
end_date: End of analysis period (inclusive).
granularity: Bucket size (day, week, month, quarter).
store_id: Filter by store ID (optional).
product_id: Filter by product ID (optional).
category: Filter by category (optional).
db: Database session.

Returns:
Time series response with points in ascending period order.

Raises:
HTTPException: If date range is invalid.
"""
# Validate date range before processing
validate_date_range(start_date, end_date)

service = AnalyticsService()
return await service.compute_timeseries(
db=db,
start_date=start_date,
end_date=end_date,
granularity=granularity,
store_id=store_id,
product_id=product_id,
category=category,
)
Loading
Loading