Pretraining trains a model on a large, varied dataset so it learns general patterns before being applied to a specific task. Applying that model to an unseen problem is transfer learning. section 2.3
Pretraining supports practical forecasting decisions by:
- Using learned patterns when a target series lacks enough history, such as forecasting sales for a new product. section 2.3
- Depending on dataset variety, frequency coverage, and prediction horizons: a model generalizes less reliably outside the conditions it saw during training. section 4.1.2
- Balancing model size, training steps, inference speed, memory, computational power, and predictive performance. section 6.1.2
Pretraining N-BEATS
A monthly N-BEATS model can take 24 months of input to predict the next 12 months, train for 1,000 gradient-update steps on the monthly M3 dataset, and then be saved for transfer learning. section 2.4
Pretraining does not guarantee strong zero-shot forecasts. Lag-Llama, for example, was trained on daily-to-minute data and horizons of 24–60 steps, so weekly, monthly, quarterly, or longer-horizon use may require fine-tuning. section 4.1.2
Listing 2.3 Initializing the N-BEATS model
from neuralforecast.core import NeuralForecast
from neuralforecast.models import NBEATS
horizon = 12
models = [
NBEATS(
input_size=2*horizon, #1
h=horizon, #2
max_steps=1000) #3
]
Inspect the N-BEATS configuration showing how input size, forecast horizon, and maximum training steps operationalize pretraining. section 2.4
