Skip to content

[ENH] auto-generate model overview table from registry tags#2236

Open
IgnazioDS wants to merge 4 commits intosktime:mainfrom
IgnazioDS:feat/automated-model-overview
Open

[ENH] auto-generate model overview table from registry tags#2236
IgnazioDS wants to merge 4 commits intosktime:mainfrom
IgnazioDS:feat/automated-model-overview

Conversation

@IgnazioDS
Copy link
Copy Markdown

Summary

Replaces the manually maintained CSV comparison table in models.rst with an auto-generated table built from model _tags metadata via the all_objects() registry.

  • New Sphinx extension (_ext/model_overview.py) with .. model-overview:: directive
  • Queries the registry at docs build time to extract capability tags
  • Renders an RST grid table with the same columns as the old manual table
  • Adding/updating models in the docs is now automatic — just set the right _tags

Motivation

The current table (lines 24-35 of models.rst) is manually maintained and doesn't include newer models like DLinear, TimeXer, Samformer, TFT v2, or TiDE v2. Every new model requires a manual edit to the docs, which is error-prone and easy to forget.

How It Works

.. model-overview::

At build time, this directive:

  1. Calls all_objects(return_tags=[...]) to discover all registered models
  2. Extracts capability:exogenous, capability:multivariate, capability:pred_int, info:pred_type, info:y_type, info:compute
  3. Renders a grid table with cross-references to model classes

Columns

Column Source Tag
Name info:name + module path for :py:class: cross-ref
Covariates capability:exogenous
Multiple targets capability:multivariate
Regression "numeric" in info:y_type
Classification "category" in info:y_type
Probabilistic "distr" in info:pred_type
Prediction intervals capability:pred_int
Compute (1-5) info:compute

Test Plan

  • sphinx-build completes without errors
  • Generated table includes all registered models (RecurrentNetwork, DecoderMLP, NBeats, NBeatsKAN, NHiTS, DeepAR, TFT, TiDE, xLSTM, DLinear, TimeXer, Samformer, etc.)
  • Cross-references (:py:class:) resolve correctly
  • New models added with proper _tags automatically appear in the table

Addresses #1934

Replace the manually maintained CSV table in models.rst with an
auto-generated table built from model _tags metadata. This ensures
the documentation always reflects the actual registered models.

Implementation:
- New Sphinx extension (_ext/model_overview.py) with `.. model-overview::`
  directive that queries all_objects() with return_tags
- Extracts capability tags (exogenous, multivariate, pred_int, etc.)
  and renders an RST grid table
- Registers extension in conf.py

Adding a new model to the docs is now automatic — just register it with
proper _tags and the table updates on next docs build.

Addresses sktime#1934
@IgnazioDS
Copy link
Copy Markdown
Author

@fkiraly Hi! This implements #1934 — auto-generating the model overview table from the registry using a custom Sphinx directive. Fixed the ruff lint issue in a follow-up commit. CI should be green now (readthedocs build still pending).

The approach mirrors how sktime handles automated docs generation — queries all_objects() with return_tags at build time and renders an RST grid table. Happy to adjust if you'd prefer a different format or additional tag columns!

@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 26, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@81d1554). Learn more about missing BASE report.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2236   +/-   ##
=======================================
  Coverage        ?   86.93%           
=======================================
  Files           ?      166           
  Lines           ?     9765           
  Branches        ?        0           
=======================================
  Hits            ?     8489           
  Misses          ?     1276           
  Partials        ?        0           
Flag Coverage Δ
cpu 86.93% <ø> (?)
pytest 86.93% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Member

@phoeenniixx phoeenniixx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot! This looks really nice!
I have one recommendation:
Would it be possible that the table shows:

  • Model_name if Model_pkg
  • Model_v2 if Model_pkg_v2

This would be helpful for the users to differentiate between the "stable" v1 models and experimental v2 models. Also, pls donot show pkg name in the table, just the model name, i.e, if the name of the pkg class is DecoderMLP_pkg it should display DecoderMLP and if the name is DLinear_pkg_v2 then it should show DLinear_v2.

OR
Would it be better to keep just the model names, but you add a new column where you add the v1, v2 tag. And explain this column below the table?

You could easily differentiate between v1, v2 pkg using the tag "object_type", where v1 models use ["forecaster_pytorch", "forecaster_pytorch_v1"] and v2 models use "forecaster_pytorch_v2". See here

What are your thoughts? @IgnazioDS
FYI @fkiraly @PranavBhatP

@phoeenniixx phoeenniixx changed the title feat: auto-generate model overview table from registry tags [ENH] auto-generate model overview table from registry tags Mar 29, 2026
@phoeenniixx phoeenniixx added documentation Improvements or additions to documentation enhancement New feature or request labels Mar 29, 2026
- Strip _pkg_v2 suffix and display as ModelName_v2 (e.g. TFT_pkg_v2 -> TFT_v2)
- Strip _pkg suffix and display as ModelName (e.g. TFT_pkg -> TFT)
- Add Version column using object_type tag: v1 for forecaster_pytorch/forecaster_pytorch_v1, v2 for forecaster_pytorch_v2
- Remove raw pkg class name from display
@IgnazioDS
Copy link
Copy Markdown
Author

Addressed your feedback @phoeenniixx:

  • Model names now strip _pkg and _pkg_v2 suffixes (e.g., TFT_pkgTFT, DLinear_pkg_v2DLinear_v2)
  • Added a Version column using the object_type tag to clearly label v1 vs v2 models
  • Removed the raw class name column entirely

Thanks for the clear guidance! Let me know if you'd prefer the alternative approach (separate v1/v2 column with footnote).

Copy link
Copy Markdown
Member

@phoeenniixx phoeenniixx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very Nice! It is much better thanks!
I looked at the new page, it looks perfect!
But the links to the models still open the pkg class of the respective model, I think it should lead to the model's implementation...
Also, I think it would be better if you could add a footnote explaining what "version" mean here and that v2 is still experimental and link the issue #1736 and roadmap issue #1993?

Also, I would suggest to keep your branch updated, that prevents any merge conflicts!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants