chapter three

3 TimeGPT

 

This chapter covers

  • Defining generative models
  • Exploring the architecture and inner workings of TimeGPT
  • Forecasting with TimeGPT
  • Anomaly detection with TimeGPT

In October 2023, the first foundation model for time series forecasting was released by Azul Garza, Cristian Challu and Max Mergenthaler-Canseco: TimeGPT [1]. This marked an important shift in paradigm for the field of forecasting, as most practitioners were building and tweaking a single model per scenario. With TimeGPT, a single model can be used for forecasting a wide range of time series data, and it can even perform anomaly detection.

Of course, we recognize the popular acronym GPT, which stands for generative pretrained Transformer.

Generative pretrained transformer

A generative pretrained Transformer, or GPT, uses the Transformer architecture and it is pretrained on a vast amount of data. The model is generative because it can take an input sequence and generate another sequence. The output sequence is produced autoregressively, meaning that each element in the sequence depends on its previous element. This allows the model to generate output sequence of arbitrary length.

We first take some time to define a generative model and move on to exploring the inner workings of TimeGPT. Then, we get hands-on experience with forecasting and anomaly detection with TimeGPT.

3.1 Defining generative pretrained Transformers

3.2 Exploring TimeGPT

3.2.1 Training TimeGPT

3.2.2 Uncertainty quantification in TimeGPT

3.3 Forecasting with TimeGPT

3.3.1 Fine-tuning with TimeGPT

3.3.2 Forecasting with exogenous variables

3.3.3 Cross-validation with TimeGPT

3.3.4 Forecasting on a long horizon with TimeGPT

3.4 Anomaly detection with TimeGPT

3.5 Next steps

3.6 Summary

3.7 References