6 Moirai: A universal forecasting transformer

 

This chapter covers

  • Discovering the Moirai model
  • Zero-shot forecasting with Moirai
  • Forecasting with exogenous features
  • Anomaly detection with Moirai

Until now, TimeGPT is the only foundation model that supports exogenous features. As a reminder, exogenous features are external factors that impact our series, like holidays for example. At the time of writing, both Lag-Llama and Chronos can perform only univariate forecasting and do not support exogenous features.

Yet, this is a critical aspect to forecasting, as future values of time series are often influenced by external factors. For example, the sales of flowers and chocolates are likely influenced by holidays, so having a way to feed that information to our models will result in better performances.

This is where Moirai comes in, as one of the first open-source foundation models to support exogenous features out of the box. As such, it is ideal to use when we have external variables, and it is also accessible via a Python package.

Here, we first explore the architecture of Moirai, to understand how it processes the input series. Also, we discover how the model was pretrained, as the quality of a foundation model relies heavily on its pretraining protocol. Then, we apply the model in our sales forecasting scenario and for anomaly detection.

6.1 Exploring Moirai

6.1.1 Architecture of Moirai

6.1.2 Pretraining Moirai

6.1.3 Selecting the appropriate model

6.2 Discovering Moirai-MoE

6.2.1 Patching and embedding

6.2.2 Decoder-only transformer

6.2.3 Pretraining Moirai-MoE

6.3 Forecasting with Moirai

6.3.1 Zero-shot forecasting with Moirai

6.3.2 Cross-validation with Moirai

6.3.3 Forecasting with exogenous features

6.4 Anomaly detection with Moirai

6.5 Next steps

6.6 Summary