[New Model] XLinear Model/ MARS algorithm

[Rud854]

I have reached out to the author of the XLinear model to work on adding this to our ecosystem.
Zaiwen/XLinear#2

I am rather new to editing models towards Github, but once it is approved, I will try to send a draft. I will admit I might make mistakes and will need help in implementing this.

I also was looking at MARs algorithm as well to implement in Darts, would a raw implementation of MARs where you manually write it or would I have to get permission from the original author in Python or R.
https://github.com/scikit-learn-contrib/py-earth

[Rud854]

Forgot to mention XLinear Models work with multivariate/ univariate forecasting with exogenous variables though from the code I have not entirely determined if it is past/ future covariates
https://arxiv.org/abs/2601.09237 link to the paper for citation

[daidahao]

@Rud854

Thank you for your suggestions and willingness to contribute.

I am not maintainer of Darts but here are my two cents:

I think XLinear is already included in the latest NeuralForecast, so Darts users can use it directly via NeuralForecastModel. However, I think XLinear is an efficient, versatile (covariate support) model, so having a native implementation in Darts would be nice addition.

Because Zaiwen/XLinear does not have a license listed, it is generally assumed to be "all rights reserved" and NOT free. It would be nice if the authors could clarify whether this is the case here. cc: @Xinyangyyyyy

We could only include the code if it is compatible with Apache 2.0. Otherwise, we would need to reimplement the model from scratch.

I am not familiar with MARS algorithm you mentioned. Is that another forecasting model? Or a proprietary software as referenced on py-earth repo? If it is the latter, I am not sure how it can be included in Darts.

[Rud854]

Oh I am so sorry, I did not notice the XLinear was on Neural Forecast. But sure, I am willing to do that though I will admit my coding skills are a work in progress.

As for the MARs I am trying to build a rough draft currently as I understand that a bit more than Pytorch. But yes, Multivariate Adaptive Regressive Splines have been used in time series regression models. From what I understand, the fast algorithm itself is proprietary, but the general formulas/ basic usage seems possible. I will try to send the draft of the model I am building but I haven't converted to the Darts ecosystem, till I understand the ecosystem more like the raw code used to make Darts forecasting models.

https://www.sciencedirect.com/science/article/pii/S2405844023079380 (particularly for CPI forecasting)

[Rud854]

This is a rough draft. This does not have the Darts beautiful API yet. I will need help to convert this into a viable Darts model. I built this MARS like model to the best of my ability and there may be mistakes. This was handmade.

@daidahao and @dennisbader
MARs Model.py

Canada.xlsx

I used this file for my code test.

I will wait for approval from the author for XLinear to attempt anything. This is the first time I am actually trying to author something instead of implementing it. Thank you for your patience, as a suggestion add this Canada file to Darts Dataset. Consider this my initial gift to Darts.

[Rud854]

As for the background of the Canada file.

I have taken the US, EU, UK, and Canadian CPI and normalized the data to Jan 1996 = 100.

US CPI - CPIAUCSL (taken from FRED) https://fred.stlouisfed.org/series/CPIAUCSL#
EU CPI - HICP Index (taken from EuroStat) https://ec.europa.eu/eurostat/web/products-euro-indicators/w/2-31032026-ap
UK CPIH Index (taken from the Office of National Statistics) https://www.cpih.co.uk/
Canada CPI Index (taken from the Statisque la Canada) https://www.statcan.gc.ca/en/subjects-start/prices_and_price_indexes/consumer_price_indexes

[daidahao]

As for the background of the Canada file.

@Rud854 That is an interesting, new dataset! Would you mind opening a separate issue for the dataset only so everyone can better evaluate the proposal and it won't be accidentally closed when completing a PR.

As for the MARS, I will try to take a closer look and provide more comments here. Thanks again for the contribution.

[Rud854]

Alright

[daidahao]

Hi @Rud854, thank you for the wait.

I've taken a closer look at your MARs.Model.py and some reference. My understanding is that your MARS model solves a regression problem: In the example you provided, you are predicting US CPI with contemporary Canada, EU, and UK CPIs. However, time series regression is currently not a formal use case of Darts (see #3016 (comment), #3031), which means regression is only supported in limited cases: SKLearnModel with lags=None and lags_future_covariates=[0].

If we were to convert your MARS model into a Darts model, then we have to do either one of these things:

• Implement MARS as a forecasting model: Use tabularization.py to convert time series data into tabular format, and apply MARS on the tabular data, i.e., treating lagged targets, covariates as features X to predict future targets y. However, I am not aware of existing forecasting libraries that implement MARS (please correct me if I'm wrong). Core question to answer before proceeding: Could a tabulated MARS become an effective forecasting model? (I did some literature search but they mostly narrow case studies.)
• Implement MARS as a regression model: Develop a unified regression API in Darts, in parallel to forecasting, classification, detection APIs. Only then can we implement MARS model as a regression model. This is perhaps less desirable since regression is not on the roadmap and requires more debate and justifications.

If we could answer the core question positively, then there is certainly value in adding MARS as a forecasting model. Otherwise, I recommend we focus on well-tested models like XLinear here.

cc: @dennisbader

Reference:

• MARS in R (parsnip)
• MARS on wikipedia

[Rud854]

I know a library called ModelTime (in R) found a way to implement MARs as a forecasting algorithm. I am not sure on
https://cran.r-project.org/web/packages/modeltime/vignettes/getting-started-with-modeltime.html

Unfortunately in Python most recommend turning the problem into a tabularized version.

Based on research, I think if tuned right MARs is quite effective for time series data (specifically with financial/economic data, though I personally have struggled to add hyper parameters to the raw model. I'm sure pyearth implements much more detailed hyper parameters.

https://www.sciencedirect.com/science/article/abs/pii/S1474034617301477
This is the closest one I focus that mentions the result of MARs used as a forecasting model here. It stands with ARIMA and SVR models in this study

May I ask what the debate is about time series regression? Is it a API problem where you won't want to replicate sktime like API.

[daidahao]

@Rud854 Thank you for the follow-up.

I've read the R package vignettes you mentioned but could not determine how it transforms MARS into a forecasting model.

In your experiment, did to apply tabularization to time series before fitting MARS model for forecasting? Or did you fit MARS without tabularization and solve a regression problem? If it is the former, how well did MARS adapt to a large number of input features (input length x components) of time series?

Regrading your last point, there is no immediate plan for supporting regression in Darts. Because it takes great efforts to support a new task, I would advise we first figure out whether this is a feature many users want, and how it would theoretically compare to contemporary alternatives. IMHO, scikit-learn already ships many regression models compatible with time series data and Darts only add limited value to it, e.g., in adding datetime attributes and performing sampling/differencing.

[dennisbader]

Hi all, and thanks for the discussion.

I don't think that we'll ever add dedicated regression models beyond the SKLearnModel support that we already have. Darts main focus is on forecasting, and regression tasks are handled already by other packages.

Regarding XLinear: we could potentially add it, but I don't see it as a high priority, since it's already available via the NeuralForecastModel.