Transfer learning

Q: How deeply does Time Series Forecasting Using Foundation Models cover transfer learning?

Explains pretraining, zero-shot prediction, and fine-tuning, but demonstrates only zero-shot application.

As explained in Time Series Forecasting Using Foundation Models

Transfer learning is a technique that applies knowledge learned by a model on one dataset or task to a different dataset or task.

Transfer learning applies a pretrained model to a dataset or task it has not seen, relying on the model’s ability to generalize knowledge from previous data. section 2.3

Use transfer learning through two main modes:

Generate zero-shot predictions by passing the new data to the pretrained model without training on it. section 2.11 section 2.5
Fine-tune the pretrained model for a few training steps so it specializes in the target task. section 2.11
Prefer it when the target dataset is too small to train a task-specific model, reducing development time and cost while data accumulates. section 2.3

Zero-shot forecasting for a new product

A forecasting model pretrained on the monthly M3 dataset can predict antidiabetic-drug prescriptions in Australia, even though it has never seen that series. Reserve the final 12 observations for testing, load the model, and call predict on the earlier observations without additional training. section 2.3 section 2.5

Listing 2.9 Zero-shot forecasting with our pretrained model

zero_shot_preds = pretrained_model.predict(input_df)
zero_shot_preds.head()

Inspect the minimal prediction step that applies a loaded pretrained model to previously unseen time-series data. section 2.5

How Time Series Forecasting Using Foundation Models covers it

Cover of Time Series Forecasting Using Foundation Models

liveBook manning.com

Read Time Series Forecasting Using Foundation Models on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Inside Deep Learning: Math, Algorithms, ModelsA training approach that adapts a pretrained model to a new task, usually by reusing learned weights and matching the original preprocessing expectations.
liveBook manning.com

LLMs in ProductionTransfer learning is a machine-learning technique that adapts knowledge learned from one task or dataset to another task or domain.
liveBook manning.com

Natural Language Processing in Action, Second EditionTransfer learning is reusing knowledge learned from one task or dataset to improve learning on another task or dataset.
liveBook manning.com

Generative AI in ActionTransfer learning adapts knowledge learned from one task or dataset to improve performance on another task.
liveBook manning.com

Knowledge Graphs and LLMs in ActionTransfer learning is a machine-learning technique that reuses patterns learned from one task or domain to improve performance on another task or domain.
liveBook manning.com

Grokking Machine LearningTransfer learning is a technique that adapts a pretrained model to a new task or dataset by continuing its training, often by modifying only some parameters.
liveBook manning.com

Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIA technique that adapts knowledge learned from one task or dataset to a related task.
liveBook manning.com

Sutskever's ListTransfer learning is a technique that adapts knowledge learned from one task or dataset to another task or dataset.
liveBook manning.com

Domain-Specific Small Language ModelsThe process of taking a model trained on one task or domain and applying it to a related task or domain, often with further tuning.
liveBook manning.com

Outlier Detection in PythonTransfer learning is a technique that adapts knowledge learned by a model on one task or dataset to another task or dataset.

Compare how 11 books explain transfer learning →

Related concepts

fine-tuningused by · Fine-tuning is adapting a pretrained machine-learning model by training it further on targeted data or objectives for a task, domain, or behavior.
pretrained modeluses · A pretrained model is a machine-learning model whose parameters were learned through prior training before it is used or adapted for a subsequent task.
embeddinguses · A numerical vector representation of an item or input that encodes features or relationships useful for computational tasks.
generalizationuses · Generalization is the extension of a pattern, model, solution, or conclusion beyond the specific examples or cases from which it was derived.
unlabeled datauses · Unlabeled data consists of examples that lack target labels or task-specific annotations.
causal invarianceused by · Causal invariance is the stability of a causal relationship across changes in environments or data distributions.

This explainer was generated by Manning's knowledge graph from the text of Time Series Forecasting Using Foundation Models. The section links point to the parts of the book it draws on.