Transfer learning
Transfer learning is a machine-learning technique that reuses knowledge learned from one task, dataset, or domain to improve learning or performance on another.
Transfer Learning for Natural Language Processing
Transfer learning is a machine-learning paradigm in which knowledge acquired from one task or domain is adapted to another task or domain.
- How it is used
- Uses transfer learning to make NLP models adaptable across tasks, domains, and languages with fewer resources.
- How deep it goes
- Explains source and target settings, then works through pretrained features, final-layer adaptation, and a simulated-to-real transfer experiment.
- Examples
- Uses email spam and movie-review classification alongside a tabular column-type experiment that adds a percent class.
- What you can do after
- Apply it via ELMo, BERT, or SIMOn to adapt pretrained models and evaluate predictions on target data.
Read how Transfer Learning for Natural Language Processing explains transfer learning →
Inside Deep Learning: Math, Algorithms, Models
A training approach that adapts a pretrained model to a new task, usually by reusing learned weights and matching the original preprocessing expectations.
- How it is used
- Adapts pretrained networks to related image or text tasks when labeled data or training time is limited.
- How deep it goes
- Walks through classifier replacement, matched preprocessing, and warm-versus-frozen training; avoids training a large pretrained transformer from scratch.
- Examples
- Compares scratch, warm, and frozen ResNet classifiers for cats versus dogs, then contrasts a GRU with DistilBERT on AG News.
- What you can do after
- Apply it via torchvision.models or Hugging Face transformers, matching preprocessing and choosing trainable weights for the available labels.
Read how Inside Deep Learning: Math, Algorithms, Models explains transfer learning →
LLMs in Production
Transfer learning is a machine-learning technique that adapts knowledge learned from one task or dataset to another task or domain.
- How it is used
- Positions transfer learning between foundation-model pretraining and prompting to specialize language models for tasks or domains.
- How deep it goes
- Explains pretraining and task-specific model updates, with fine-tuning and hosted-job examples.
- Examples
- Illustrates fine-tuning with a GPT-2 text-dataset listing and an OpenAI training-file upload and job.
- What you can do after
- Evaluate when to adapt a pretrained model rather than train from scratch or try prompting without fine-tuning.
Natural Language Processing in Action, Second Edition
Transfer learning is reusing knowledge learned from one task or dataset to improve learning on another task or dataset.
- How it is used
- Uses pretrained representations to accelerate NLP models for domain-specific embeddings and downstream text classification.
- How deep it goes
- Explains initialization, vocabulary alignment, missing-token handling, fine-tuning, validation, and evaluation, but not pretrained-model training internals.
- Examples
- Uses a 4,000-token GloVe matrix and a toxic-comment BERT classifier evaluated with LRAP, accuracy, and evaluation loss.
- What you can do after
- Builds transfer-learning pipelines with PyTorch embeddings or fine-tunes pretrained BERT models for labeled text classification.
Read how Natural Language Processing in Action, Second Edition explains transfer learning →
Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI
Transfer learning is a machine-learning technique that applies knowledge learned from one task or dataset to a different task or dataset.
- How it is used
- Reduces cold-start annotation effort by adapting existing models to new tasks, and supports sampling within active learning.
- How deep it goes
- Explains frozen-layer retraining and representation reuse, with practical alternatives and limitations, without a full implementation walkthrough.
- Examples
- Uses product-to-movie sentiment transfer, ImageNet feature reuse, and a semantic segmentation annotation-budget scenario.
- What you can do after
- Evaluate transfer options for a new task by comparing layer retraining, representation reuse, and annotation effort.
Build AI Drug Discovery Pipelines
Transfer learning is a technique that adapts knowledge learned from one dataset or task to improve performance on another task.
- How it is used
- Uses pretrained protein and molecular models to support drug-discovery predictions when task-specific labels are limited.
- How deep it goes
- Explains classification-head fine-tuning and pretrained embeddings, but does not implement the transfer-learning workflow.
- Examples
- Illustrates antibody classification, protein variant-effect prediction, and small-molecule property prediction.
- What you can do after
- Evaluate/understand how pretrained representations can support downstream predictions and why they are useful with limited labels.
Read how Build AI Drug Discovery Pipelines explains transfer learning →
Generative AI in Action
Transfer learning adapts knowledge learned from one task or dataset to improve performance on another task.
- How it is used
- Positions transfer learning as a way to adapt foundation models to related tasks and domains.
- How deep it goes
- Defines knowledge reuse, contrasts fine-tuning methods, and outlines a fine-tuning workflow without detailed mathematical derivation.
- Examples
- Illustrates transfer with a model trained on English text and adapted to French text.
- What you can do after
- Evaluate/understand how to select, adapt, and test a pretrained model for a related task.
Read how Generative AI in Action explains transfer learning →
Knowledge Graphs and LLMs in Action
Transfer learning is a machine-learning technique that reuses patterns learned from one task or domain to improve performance on another task or domain.
- How it is used
- Reuses pretrained language patterns to reduce the data and computation needed for domain-specific NLP tasks.
- How deep it goes
- Explains generic pretraining, weight reuse, prompting, and fine-tuning, but does not detail optimization internals.
- Examples
- Uses masked-token prediction transferred to relation extraction, named-entity recognition, and knowledge-graph construction.
- What you can do after
- Apply pretrained models to relation extraction with less human-labeled data than training task-specific models from scratch.
Read how Knowledge Graphs and LLMs in Action explains transfer learning →
Time Series Forecasting Using Foundation Models
Transfer learning is a technique that applies knowledge learned by a model on one dataset or task to a different dataset or task.
- How it is used
- Applies pretrained forecasting models when target data is scarce or unseen.
- How deep it goes
- Explains pretraining, zero-shot prediction, and fine-tuning, but demonstrates only zero-shot application.
- Examples
- Uses an N-BEATS model pretrained on monthly M3 series to forecast Australian antidiabetic-drug prescriptions.
- What you can do after
- Apply it via NeuralForecast by loading a pretrained model and generating predictions for an unseen time series.
Read how Time Series Forecasting Using Foundation Models explains transfer learning →
Machines that Think
Transfer of learning is the reuse of knowledge or structure learned in one context to support performance in another context.
- How it is used
- Uses transfer to explain how learned language patterns support specialization and performance across contexts.
- How deep it goes
- Defines transfer and outlines pretraining followed by adaptation, without detailing an implementation procedure.
- Examples
- Illustrates domain specialization with legal and medical text and cross-task reasoning with the MathCAMPS study.
- What you can do after
- Evaluate and understand how reused structure can support new tasks without assuming human-like learning or grounded understanding.
Grokking Machine Learning
Transfer learning is a technique that adapts a pretrained model to a new task or dataset by continuing its training, often by modifying only some parameters.
- How it is used
- Adapts pretrained neural networks to complicated image datasets when training a complete model from scratch is impractical.
- How deep it goes
- Defines pretrained-network adaptation and parameter tweaking, usually at the last layer, but does not detail the training procedure.
- Examples
- Uses complicated image datasets and pretrained architectures including InceptionV3, ImageNet, ResNet, and VGG.
- What you can do after
- Apply it via InceptionV3, ImageNet, ResNet, or VGG by adapting selected parameters to new image data.
Read how Grokking Machine Learning explains transfer learning →
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
A technique that adapts knowledge learned from one task or dataset to a related task.
- How it is used
- Frames pretrained deep-learning networks as reusable starting points for related image and text tasks.
- How deep it goes
- Defines adaptation and limited additional effort, but stops short of describing fine-tuning procedures or implementation details.
- Examples
- Contrasts adapting image or text networks with the scarcity of reusable pretrained solutions for varied tabular datasets.
- What you can do after
- Evaluate/understand when pretrained deep-learning networks can transfer across related tasks and why tabular cases offer fewer reusable options.
Sutskever's List
Transfer learning is a technique that adapts knowledge learned from one task or dataset to another task or dataset.
- How it is used
- Uses an ImageNet-trained vision model as the starting point for adaptation to other datasets.
- How deep it goes
- Demonstrates fine-tuning results across datasets but gives no adaptation procedure.
- Examples
- Compares results on CIFAR-100, CIFAR-10, Stanford Cars, Oxford Pets, Food-101, FGVC Aircraft, and Birdsnap.
- What you can do after
- Evaluate transfer results by comparing fine-tuned model performance across vision datasets.
Domain-Specific Small Language Models
The process of taking a model trained on one task or domain and applying it to a related task or domain, often with further tuning.
- How it is used
- Specializes pretrained models for related domains and tasks using domain-specific data.
- How deep it goes
- Defines adaptation and compares tuning strategies and tradeoffs, but does not detail their implementation.
- Examples
- Uses molecule descriptions and SMILES structures for molecular generation and description.
- What you can do after
- Apply pretrained models to related specialized tasks by tuning them on domain-specific data.
Read how Domain-Specific Small Language Models explains transfer learning →
Build an Advanced RAG Application (From Scratch)
Transfer learning is a machine-learning technique that adapts knowledge learned for one task or domain as the starting point for another.
- How it is used
- Uses pre-trained models as starting points for related but different tasks, improving performance and reducing training time.
- How deep it goes
- Defines pre-training transfer and its benefits, but stops short of implementation steps, fine-tuning details, or quantitative trade-offs.
- Examples
- Uses Transformer pre-training on vast amounts of unlabeled text as the transfer-learning scenario.
- What you can do after
- Evaluate transfer learning as a way to improve performance and reduce training time when adapting a pre-trained model.
Read how Build an Advanced RAG Application (From Scratch) explains transfer learning →
Outlier Detection in Python
Transfer learning is a technique that adapts knowledge learned by a model on one task or dataset to another task or dataset.
- How it is used
- Uses learned image representations to reduce the effort of training models for related tasks, especially in deep learning-based outlier detection.
- How deep it goes
- Explains foundation models, embeddings, fine-tuning, self-supervision, and efficiency benefits, but stops short of implementation code.
- Examples
- Uses cat-versus-dog classification, dog-type classification, image rotations, and comparisons between image and tabular data.
- What you can do after
- Apply it via fine-tuning by reusing embeddings and retraining a final or added layer for a related task.
Read how Outlier Detection in Python explains transfer learning →
Also covered in
- Machine Learning System Design
- Learn Generative AI with PyTorch
- Timeless Algorithms, The Foundational IdeasIn early access (MEAP)
- Building Specialized AI SystemsIn early access (MEAP)
- Designing Deep Learning Systems: A guide for software engineers
- Causal AI
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- transformer architectureA transformer is a neural-network architecture that uses attention-based interactions and position-wise transformations to model relationships among elements of sequences or other structured inputs.
- foundation modelA foundation model is a broadly trained machine-learning model whose learned capabilities can be reused or adapted for a range of downstream tasks.
- natural language processingNatural language processing is the field of computing concerned with computational methods for analyzing, interpreting, transforming, and generating human language in text or speech.
Related concepts
- fine-tuningused by · Fine-tuning is the continued training of a pretrained model on targeted data or objectives to adapt its parameters or behavior to a task, domain, or use.
- pretrained modeluses · A pretrained model is a machine-learning model whose parameters were learned through prior training before it is used for inference or adapted to a later task.
- embeddinguses · An embedding is a numerical vector representation of an item in a feature space, encoding selected properties or relationships for computational tasks such as comparison or retrieval.
- adaptive transfer learningused by · A transfer learning approach that adapts during an active learning process.
- causal invarianceused by · Causal invariance is the stability of a causal relationship across changes in environments or data distributions.
- cold start problemaddresses · A cold-start problem is the difficulty or cost of beginning operation without sufficient accumulated data, interaction history, or initialized runtime state.
Questions about transfer learning
What is transfer learning?
Transfer learning is a machine-learning technique that reuses knowledge learned from one task, dataset, or domain to improve learning or performance on another.
Which Manning books explain transfer learning?
Transfer Learning for Natural Language Processing, Inside Deep Learning: Math, Algorithms, Models, LLMs in Production, Natural Language Processing in Action, Second Edition, Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI, Build AI Drug Discovery Pipelines, Generative AI in Action, Knowledge Graphs and LLMs in Action and 8 more.
What should I know before learning transfer learning?
Manning books that teach transfer learning usually introduce language model, transformer architecture, foundation model and natural language processing first.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.















