liveBook · Manning

Out-of-sample forecast

As explained in Time Series Forecasting Using Foundation Models

An out-of-sample forecast is a prediction for future observations that were not included in the model input or fitting data.

An out-of-sample forecast predicts future time steps that were excluded from the model input or fitting data, rather than reproducing observations already seen. section 4.2.2

Eight-week store-sales forecast

Lag-Llama produces out-of-sample predictions for the next eight weeks of sales at four Walmart stores, including 95% prediction intervals.

Listing 4.2 Defining a function to get predictions from Lag-Llama
def get_lag_llama_predictions(dataset,
                              prediction_length,
                              device,
                              context_length=32,
                              num_samples=100):

    ckpt = torch.load("lag-llama.ckpt",
    ➥map_location=device)  #1
    estimator_args = ckpt["hyper_parameters"]["model_kwargs"]

    estimator = LagLlamaEstimator(
        ckpt_path="lag-llama.ckpt",
        prediction_length=prediction_length,
        context_length=context_length,
        input_size=estimator_args["input_size"],  #2
        n_layer=estimator_args["n_layer"],
        n_embd_per_head=estimator_args["n_embd_per_head"],
        n_head=estimator_args["n_head"],
        scaling=estimator_args["scaling"],
        time_feat=estimator_args["time_feat"],
        batch_size=1,
        num_parallel_samples=num_samples,
        device=device,
    )

    lightning_module =
    ➥estimator.create_lightning_module()  #3
    transformation =
    ➥estimator.create_transformation()  #4
    predictor = estimator.create_predictor(
    ➥transformation, lightning_module)  #5

    forecasts = predictor.predict(  #6
        dataset=dataset,
    )

    forecasts = list(forecasts)  #7

    return forecasts

Inspect the prediction function that loads Lag-Llama parameters, configures the context and horizon, and returns forecast samples. section 4.2.2

How Time Series Forecasting Using Foundation Models covers it

How deep it goes
Explains dataset preparation, horizon selection, inference, and interval formatting, but defers formal evaluation metrics.
Examples
Uses Walmart weekly sales, four stores, an eight-week horizon, forecast samples, medians, and 95% prediction intervals.
What it assumes
Assumes Python, Pandas, PyTorch, GluonTS, and a locally available Lag-Llama checkpoint.
What you can do after
Builds future forecasts with Lag-Llama and formats sampled outputs as median predictions with confidence intervals.
How it connects
Connects time-series forecasting with zero-shot learning, GluonTS, Lag-Llama, probabilistic forecasting, and prediction intervals.

Also explained in

Compare how 2 books explain out-of-sample forecast →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Time Series Forecasting Using Foundation Models. The section links point to the parts of the book it draws on.