A context window is the fixed number of preceding sequence elements a forecasting model can consider when processing an input. In Lag-Llama, a context length of 32 means the model uses the previous 32 time steps. section 4.2.2
The window controls how much history reaches the model. Longer windows can capture stable patterns, while shorter windows can emphasize recent anomalies; no single length works for every dataset. section 4.2.3
In Lag-Llama, each token combines lagged values across the context length with date-time features. This richer input can increase inference time because more information must be processed. section 4.1.1
Tune context length with an explicit forecasting experiment:
- Choose a forecast horizon and reserve historical observations for testing. section 4.2.3
- Run nonoverlapping cross-validation windows for candidate lengths such as 16, 32, and 64 time steps. section 4.2.3
- Compare forecasts with MAE and sMAPE, where lower values indicate better performance. section 4.2.3
Walmart weekly-sales experiment
Using one Walmart store, four eight-week validation windows compared context lengths of 16, 32, and 64 in Lag-Llama. The 64-step window produced the best reported MAE and sMAPE results, so it was reused for fine-tuning. section 4.2.3 section 4.3.1
Listing 4.12 Making predictions with different context lengths
fcsts_16 = cross_validation_lagllama(
df=df,
h=8,
n_windows=4,
device=device,
context_length=16
)
fcsts_32 = cross_validation_lagllama(
df=df,
h=8,
n_windows=4,
device=device,
context_length=32
)
fcsts_64 = cross_validation_lagllama(
df=df,
h=8,
n_windows=4,
device=device,
context_length=64
)
Inspect the candidate context lengths used in the comparison experiment. section 4.2.3
