Skip to content

feat(backtest): wire feature-aware models into the backtesting fold loop (#244)#245

Merged
w7-mgfcode merged 1 commit into
devfrom
feat/backtesting-feature-aware-folds
May 19, 2026
Merged

feat(backtest): wire feature-aware models into the backtesting fold loop (#244)#245
w7-mgfcode merged 1 commit into
devfrom
feat/backtesting-feature-aware-folds

Conversation

@w7-mgfcode
Copy link
Copy Markdown
Owner

Implements PRPs/PRP-MLZOO-B.2-feature-aware-backtesting.md. Closes #244.

Feature-aware models (regression, lightgbm — any model with
requires_features=True) are now evaluated by POST /backtesting/run and
backtest jobs, with per-fold leakage-safe X_train / X_future matrices.
This is the follow-up deferred by PRP-30 DECISIONS LOCKED #6.

What changed

  • Shared row buildersapp/shared/feature_frames/rows.py:
    build_historical_feature_rows (promoted verbatim from
    ForecastingService._assemble_regression_rows) + build_future_feature_rows,
    the leakage-critical per-fold test-window assembler. _assemble_regression_rows
    becomes a delegating shim — its load-bearing leakage spec stays green, unedited.
  • BacktestingService — resolves exogenous data once (async ExogenousFrame),
    branches the fold loop on requires_features, slices X_train from one
    historical matrix, rebuilds X_future per fold (never sliced — that would leak
    an adjacent test-day target). min_train_size >= 30 enforced loud.
  • ModelBacktestResult — additive feature_aware / exogenous_policy fields;
    _shape_backtest_result keys unchanged (frontend contract byte-stable).
  • JobService._execute_backtest — accepts regression + lightgbm.
  • Interim loud-fail test repurposed (PRP-29 chore(main): release 0.1.1 #7 / PRP-30 fix: remove CRLF line endings from pyproject.toml #6 superseded); new shared
    X_future leakage spec including a gap>0 fold.

Scope

No frontend/, scenarios/, or alembic/ change; no migration.

Validation

  • ruff + mypy --strict + pyright --strict — clean
  • unit — 1341 passed (incl. test_regression_features_leakage.py unedited)
  • integration — 20 passed (regression backtest + baselines vs real Postgres)
  • dogfood — regression backtest feature_aware:true/exogenous_policy:"observed";
    min_train<30 → 400; backtest job completes; naive baseline unaffected

🤖 Generated with Claude Code

…oop (#244)

Implements PRP-MLZOO-B.2. Feature-aware models (regression, lightgbm — any
model with requires_features=True) are now evaluated by POST /backtesting/run
and backtest jobs, with per-fold leakage-safe X_train / X_future matrices.

- Promote the historical row builder to app/shared/feature_frames/rows.py
  (build_historical_feature_rows) and add build_future_feature_rows — the
  leakage-critical per-fold test-window assembler. ForecastingService.
  _assemble_regression_rows becomes a delegating shim; its leakage spec
  (test_regression_features_leakage.py) stays green, unedited.
- BacktestingService: resolve exogenous data once async (ExogenousFrame),
  branch the fold loop on requires_features, slice X_train from one historical
  matrix, rebuild X_future per fold. min_train_size >= 30 enforced loud.
- ModelBacktestResult gains additive feature_aware / exogenous_policy fields;
  _shape_backtest_result keys unchanged (frontend contract byte-stable).
- JobService._execute_backtest accepts regression + lightgbm.
- Repurpose the interim loud-fail test (PRP-29 #7 / PRP-30 #6 superseded);
  add the shared X_future leakage spec incl. a gap>0 fold.
Copy link
Copy Markdown

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @w7-mgfcode, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 19, 2026

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: bdc4902e-1853-4012-8698-e8697dedf3f5

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/backtesting-feature-aware-folds

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@w7-mgfcode w7-mgfcode merged commit 5a65e35 into dev May 19, 2026
10 of 12 checks passed
@w7-mgfcode w7-mgfcode deleted the feat/backtesting-feature-aware-folds branch May 20, 2026 03:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant