chapter six

6 Moirai: A universal forecasting transformer

 

This chapter covers

  • Discovering the Moirai model
  • Zero-shot forecasting with Moirai
  • Forecasting with exogenous features
  • Detecting anomalies with Moirai

Until now, TimeGPT is the only foundation model covered in this book so far that supports exogenous features—external factors that affect our series, such as holidays. At this writing, Lag-Llama and Chronos can perform only univariate forecasting; they don’t support exogenous features. Yet this support is a critical aspect of forecasting because future values of time series are often influenced by external factors. The sales of flowers and chocolates, for example, are likely influenced by holidays, so having a way to feed that information to our models will result in better performance.

Moirai is one of the first open source foundation models to support exogenous features out of the box. It’s ideal to use when we have external variables, and it’s also accessible via a Python package.

This chapter explores the architecture of Moirai to show how it processes the input series. We discover that the model was pretrained because the quality of a foundation model relies heavily on its pretraining protocol. Then we apply the model in our sales forecasting scenario and to anomaly detection.

6.1 Exploring Moirai

6.1.1 Viewing the architecture of Moirai

6.1.2 Pretraining Moirai

6.1.3 Selecting the appropriate model

6.2 Discovering Moirai-MoE

6.2.1 Patching and embedding

6.2.2 Studying the decoder-only transformer

6.2.3 Pretraining Moirai-MoE

6.3 Forecasting with Moirai

6.3.1 Zero-shot forecasting with Moirai

6.3.2 Cross-validation with Moirai

6.3.3 Forecasting with exogenous features

6.4 Detecting anomalies with Moirai

6.5 Next steps

Summary