Tokenization converts time-series input into discrete units that sequence models can process. In forecasting, it takes two distinct forms: Lag-Llama packages lagged values and date-time features into tokens, while Chronos scales and quantizes real-valued observations into a finite vocabulary. section 4.1.1 section 5.2
Common forecasting tokenization workflows include:
- Lag-Llama includes past values over a fixed context length and date-time features such as hour of day, day of week, and quarter of year. Their changing patterns help indicate the series frequency. section 4.1.1
- Chronos first applies mean scaling by dividing each point by the mean of absolute values, preserving the series shape and meaningful zeros. section 5.3
- Chronos then quantizes scaled values into a finite set of bins. More bins retain precision; too few bins produce excessive approximation. section 5.3
- At prediction time, Chronos reverses tokenization by dequantizing and unscaling generated tokens back into time-series values. section 5.2
Hourly forecasting and bin choice
For hourly data, hour of day changes at every step, whereas day of week and day of month change less often; this pattern helps Lag-Llama infer frequency. In Chronos, a vocabulary of 100 bins retains more precision than five bins, which loses too much detail. section 4.1.1 section 5.3
Figure 4.2 Tokenization process in Lag-Llama. Here, the token of the current time step includes lagged values of the series over a certain context length, as well as date-time features.
Shows how a Lag-Llama token combines lagged values with date-time features across the context window. section 4.1.1
Figure 5.2 Overview of the Chronos framework, which adapts a language model for time-series forecasting. First, the time series is tokenized, which involves mean scaling and quantization. Then the output can be used to train the language model.
Shows the Chronos pipeline from mean scaling and quantization to language-model-compatible tokens. section 5.2
