Fine-tuning adapts a pretrained model to a specific dataset by training it for a limited number of parameter updates. It is a form of transfer learning that specializes a general model without repeating full pretraining. section 2.6 section 2.11
The usual strategy tunes only parameters near the output, reducing time and computation. Tuning every layer can improve adaptation but increases cost and may cause overfitting, weakening generalization. section 2.6
A practical fine-tuning workflow is:
- Create training and test data for the target series, then generate zero-shot predictions as a baseline. section 3.4.1
- Choose a small training budget; ten steps is recommended as a starting point for TimeGPT, while Lag-Llama uses epochs. section 3.4.1 section 4.3.1
- Select a loss function such as MAE, MSE, RMSE, MAPE, or sMAPE according to the error behavior that matters. section 3.4.1
- Increase tuning depth gradually and monitor a validation set because larger step counts or deeper tuning can increase prediction time and overfitting. section 3.4.3 section 3.4.1
- Compare the adapted model with the baseline using explicit metrics and, where possible, cross-validation. section 3.4.2 section 5.7.5
TimeGPT on weekly store sales
A zero-shot TimeGPT forecast for Store 1 produced an MAE of 75,237 and sMAPE of 2.41%. Ten fine-tuning steps with MAE reduced these to 70,822 and 2.28%; tuning more parameters reduced them further to 63,544 and 2.04%. section 3.4.2 section 3.4.3
Local model training
Fine-tuning Lag-Llama uses a local training setup with a T4 GPU, context length 64, learning rate 5e-4, batch size 64, and a 50-epoch limit. Fine-tuned Chronos can be loaded from its saved checkpoint and evaluated by cross-validation. section 4.3.1 section 5.7.4
Listing 3.11 Fine-tuning TimeGPT
finetune_preds = nixtla_client.forecast(
df=input_df,
h=8,
finetune_steps=10, #1
finetune_loss='mae', #2
time_col='Date',
id_col='Store',
target_col='Weekly_Sales'
)
Shows the compact API call that applies a ten-step TimeGPT adaptation with MAE on an eight-week forecast. section 3.4.1
