feat(backtest): wire feature-aware models into the backtesting fold loop (#244)#245
Conversation
…oop (#244) Implements PRP-MLZOO-B.2. Feature-aware models (regression, lightgbm — any model with requires_features=True) are now evaluated by POST /backtesting/run and backtest jobs, with per-fold leakage-safe X_train / X_future matrices. - Promote the historical row builder to app/shared/feature_frames/rows.py (build_historical_feature_rows) and add build_future_feature_rows — the leakage-critical per-fold test-window assembler. ForecastingService. _assemble_regression_rows becomes a delegating shim; its leakage spec (test_regression_features_leakage.py) stays green, unedited. - BacktestingService: resolve exogenous data once async (ExogenousFrame), branch the fold loop on requires_features, slice X_train from one historical matrix, rebuild X_future per fold. min_train_size >= 30 enforced loud. - ModelBacktestResult gains additive feature_aware / exogenous_policy fields; _shape_backtest_result keys unchanged (frontend contract byte-stable). - JobService._execute_backtest accepts regression + lightgbm. - Repurpose the interim loud-fail test (PRP-29 #7 / PRP-30 #6 superseded); add the shared X_future leakage spec incl. a gap>0 fold.
There was a problem hiding this comment.
Sorry @w7-mgfcode, you have reached your weekly rate limit of 500000 diff characters.
Please try again later or upgrade to continue using Sourcery
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Implements
PRPs/PRP-MLZOO-B.2-feature-aware-backtesting.md. Closes #244.Feature-aware models (
regression,lightgbm— any model withrequires_features=True) are now evaluated byPOST /backtesting/runandbacktestjobs, with per-fold leakage-safeX_train/X_futurematrices.This is the follow-up deferred by PRP-30 DECISIONS LOCKED #6.
What changed
app/shared/feature_frames/rows.py:build_historical_feature_rows(promoted verbatim fromForecastingService._assemble_regression_rows) +build_future_feature_rows,the leakage-critical per-fold test-window assembler.
_assemble_regression_rowsbecomes a delegating shim — its load-bearing leakage spec stays green, unedited.
BacktestingService— resolves exogenous data once (asyncExogenousFrame),branches the fold loop on
requires_features, slicesX_trainfrom onehistorical matrix, rebuilds
X_futureper fold (never sliced — that would leakan adjacent test-day target).
min_train_size >= 30enforced loud.ModelBacktestResult— additivefeature_aware/exogenous_policyfields;_shape_backtest_resultkeys unchanged (frontend contract byte-stable).JobService._execute_backtest— acceptsregression+lightgbm.X_futureleakage spec including agap>0fold.Scope
No
frontend/,scenarios/, oralembic/change; no migration.Validation
--strict+ pyright--strict— cleantest_regression_features_leakage.pyunedited)feature_aware:true/exogenous_policy:"observed";min_train<30→ 400; backtest job completes; naive baseline unaffected🤖 Generated with Claude Code