Transfer learning
Transfer learning is a machine-learning technique that reuses knowledge learned from one task, domain, or dataset to improve learning or performance on another.
Inside Deep Learning: Math, Algorithms, Models
A training approach that adapts a pretrained model to a new task, usually by reusing learned weights and matching the original preprocessing expectations.
- How it is used
- Uses pretrained models to build accurate target-task models with less labeled data, compute, and training time.
- How deep it goes
- Explains weight reuse, output-layer surgery, preprocessing, warm starts, frozen layers, dataset-size tradeoffs, and transformer adaptation.
- Examples
- Demonstrates cat-versus-dog ResNet classification, ImageNet filter visualization, tiny labeled datasets, AG News, BERT, and DistilBERT.
- What you can do after
- Apply it via PyTorch, torchvision, and Hugging Face by replacing output layers, matching tokenization or normalization, and selecting warm or frozen training.
Read how Inside Deep Learning: Math, Algorithms, Models explains transfer learning →
LLMs in Production
Transfer learning is a machine-learning technique that adapts knowledge learned from one task or dataset to another task or domain.
- How it is used
- Adapts pretrained language models to specific tasks or domains without repeating general-purpose pretraining.
- How deep it goes
- Defines source and target domains, explains pretraining followed by finetuning, and contrasts training costs without detailing weight-update internals.
- Examples
- Uses BERT task adaptation, sentiment analysis, text classification, question answering, and a finetuning-versus-scratch comparison.
- What you can do after
- Apply it via OpenAI fine-tuning jobs or a pretrained model, using target-specific data rather than training from scratch.
Natural Language Processing in Action, Second Edition
Transfer learning is reusing knowledge learned from one task or dataset to improve learning on another task or dataset.
- How it is used
- Uses pretrained representations to accelerate NLP models for domain-specific embeddings and downstream text classification.
- How deep it goes
- Explains initialization, vocabulary alignment, missing-token handling, fine-tuning, validation, and evaluation, but not pretrained-model training internals.
- Examples
- Uses a 4,000-token GloVe matrix and a toxic-comment BERT classifier evaluated with LRAP, accuracy, and evaluation loss.
- What you can do after
- Builds transfer-learning pipelines with PyTorch embeddings or fine-tunes pretrained BERT models for labeled text classification.
Read how Natural Language Processing in Action, Second Edition explains transfer learning →
Generative AI in Action
Transfer learning adapts knowledge learned from one task or dataset to improve performance on another task.
- How it is used
- Uses previously learned model knowledge as a starting point for related tasks and foundation-model adaptation.
- How deep it goes
- Defines task-to-task reuse and situates it among adaptation methods, without providing transfer-learning-specific code or mathematical internals.
- Examples
- Illustrates adaptation from English to French text and mentions BERT adaptation for medical text across hospitals.
- What you can do after
- Evaluate/understand it by choosing a pretrained model and adaptation method, then preparing data and testing task performance before deployment.
Read how Generative AI in Action explains transfer learning →
Knowledge Graphs and LLMs in Action
Transfer learning is a machine-learning technique that reuses patterns learned from one task or domain to improve performance on another task or domain.
- How it is used
- Reuses pretrained language patterns to reduce the data and computation needed for domain-specific NLP tasks.
- How deep it goes
- Explains generic pretraining, weight reuse, prompting, and fine-tuning, but does not detail optimization internals.
- Examples
- Uses masked-token prediction transferred to relation extraction, named-entity recognition, and knowledge-graph construction.
- What you can do after
- Apply pretrained models to relation extraction with less human-labeled data than training task-specific models from scratch.
Read how Knowledge Graphs and LLMs in Action explains transfer learning →
Time Series Forecasting Using Foundation Models
Transfer learning is a technique that applies knowledge learned by a model on one dataset or task to a different dataset or task.
- How it is used
- Applies pretrained forecasting models when target data is scarce or unseen.
- How deep it goes
- Explains pretraining, zero-shot prediction, and fine-tuning, but demonstrates only zero-shot application.
- Examples
- Uses an N-BEATS model pretrained on monthly M3 series to forecast Australian antidiabetic-drug prescriptions.
- What you can do after
- Apply it via NeuralForecast by loading a pretrained model and generating predictions for an unseen time series.
Read how Time Series Forecasting Using Foundation Models explains transfer learning →
Grokking Machine Learning
Transfer learning is a technique that adapts a pretrained model to a new task or dataset by continuing its training, often by modifying only some parameters.
- How it is used
- Adapts pretrained neural networks to complicated image datasets when training a complete model from scratch is impractical.
- How deep it goes
- Defines pretrained-network adaptation and parameter tweaking, usually at the last layer, but does not detail the training procedure.
- Examples
- Uses complicated image datasets and pretrained architectures including InceptionV3, ImageNet, ResNet, and VGG.
- What you can do after
- Apply it via InceptionV3, ImageNet, ResNet, or VGG by adapting selected parameters to new image data.
Read how Grokking Machine Learning explains transfer learning →
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
A technique that adapts knowledge learned from one task or dataset to a related task.
- How it is used
- Frames pretrained deep-learning networks as reusable starting points for related image and text tasks.
- How deep it goes
- Defines adaptation and limited additional effort, but stops short of describing fine-tuning procedures or implementation details.
- Examples
- Contrasts adapting image or text networks with the scarcity of reusable pretrained solutions for varied tabular datasets.
- What you can do after
- Evaluate/understand when pretrained deep-learning networks can transfer across related tasks and why tabular cases offer fewer reusable options.
Sutskever's List
Transfer learning is a technique that adapts knowledge learned from one task or dataset to another task or dataset.
- How it is used
- Uses an ImageNet-trained vision model as the starting point for adaptation to other datasets.
- How deep it goes
- Demonstrates fine-tuning results across datasets but gives no adaptation procedure.
- Examples
- Compares results on CIFAR-100, CIFAR-10, Stanford Cars, Oxford Pets, Food-101, FGVC Aircraft, and Birdsnap.
- What you can do after
- Evaluate transfer results by comparing fine-tuned model performance across vision datasets.
Domain-Specific Small Language Models
The process of taking a model trained on one task or domain and applying it to a related task or domain, often with further tuning.
- How it is used
- Adapts pretrained language models for specialized domains, tasks, privacy requirements, and regulated applications.
- How deep it goes
- Defines transfer learning and compares full, partial, and parameter-efficient tuning, without detailing implementation procedures.
- Examples
- Uses molecule descriptions and SMILES structures for generation and captioning.
- What you can do after
- Apply domain-specific data to select a specialization strategy while weighing accuracy, computation, overfitting, and forgetting.
Read how Domain-Specific Small Language Models explains transfer learning →
Outlier Detection in Python
Transfer learning is a technique that adapts knowledge learned by a model on one task or dataset to another task or dataset.
- How it is used
- Uses learned image representations to reduce the effort of training models for related tasks, especially in deep learning-based outlier detection.
- How deep it goes
- Explains foundation models, embeddings, fine-tuning, self-supervision, and efficiency benefits, but stops short of implementation code.
- Examples
- Uses cat-versus-dog classification, dog-type classification, image rotations, and comparisons between image and tabular data.
- What you can do after
- Apply it via fine-tuning by reusing embeddings and retraining a final or added layer for a related task.
Read how Outlier Detection in Python explains transfer learning →
Related concepts
- fine-tuningused by · Fine-tuning is adapting a pretrained machine-learning model by training it further on targeted data or objectives for a task, domain, or behavior.
- pretrained modeluses · A pretrained model is a machine-learning model whose parameters were learned through prior training before it is used or adapted for a subsequent task.
- embeddinguses · A numerical vector representation of an item or input that encodes features or relationships useful for computational tasks.
- generalizationuses · Generalization is the extension of a pattern, model, solution, or conclusion beyond the specific examples or cases from which it was derived.
- unlabeled datauses · Unlabeled data consists of examples that lack target labels or task-specific annotations.
- causal invarianceused by · Causal invariance is the stability of a causal relationship across changes in environments or data distributions.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.