Transfer learning

As explained in Knowledge Graphs and LLMs in Action

Transfer learning is a machine-learning technique that reuses patterns learned from one task or domain to improve performance on another task or domain.

Transfer learning reuses patterns or weights learned from a generic task to improve a different task, avoiding training a separate model from scratch. section 1.2 section 5.3

A practical transfer-learning pipeline typically works as follows:

• Pretrain a language model on a generic task, such as predicting randomly masked tokens, using a large text corpus. section 1.2 section 5.3
• Reuse the resulting model or its learned weights for a specific task, such as relation extraction from domain text. section 1.2 section 5.3
• Customize the model with prompting or fine-tuning when the generic capabilities are insufficient, while preparing task-specific data as needed. section 5.3 section 5.3.4

Knowledge-graph construction

A pretrained Transformer-based LLM can transfer linguistic patterns learned from masked-token prediction to named-entity recognition and relation extraction, producing structured knowledge from unstructured text with less human-labeled data. section 5.3 section 5.3.4

Figure 1.2 In transfer learning, a model (or part of it) trained on a specific task is used as part of the training and predictions for another task.

Use the figure to compare the originating task, transferred model component, and target task or prediction. section 1.2

How Knowledge Graphs and LLMs in Action covers it

How deep it goes

Explains generic pretraining, weight reuse, prompting, and fine-tuning, but does not detail optimization internals.

Examples

Uses masked-token prediction transferred to relation extraction, named-entity recognition, and knowledge-graph construction.

What it assumes

Assumes familiarity with machine-learning tasks and supervised learning while defining transfer from generic to specific tasks.

What you can do after

Apply pretrained models to relation extraction with less human-labeled data than training task-specific models from scratch.

How it connects

Connects transfer learning with large language models, Transformer architecture, prompt engineering, fine-tuning, relation extraction, and knowledge graphs.

liveBookmanning.com

Read Knowledge Graphs and LLMs in Action on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Inside Deep Learning: Math, Algorithms, ModelsA training approach that adapts a pretrained model to a new task, usually by reusing learned weights and matching the original preprocessing expectations.
• 

  liveBookmanning.com
  LLMs in ProductionTransfer learning is a machine-learning technique that adapts knowledge learned from one task or dataset to another task or domain.
• 

  liveBookmanning.com
  Natural Language Processing in Action, Second EditionTransfer learning is reusing knowledge learned from one task or dataset to improve learning on another task or dataset.
• 

  liveBookmanning.com
  Generative AI in ActionTransfer learning adapts knowledge learned from one task or dataset to improve performance on another task.
• 

  liveBookmanning.com
  Time Series Forecasting Using Foundation ModelsTransfer learning is a technique that applies knowledge learned by a model on one dataset or task to a different dataset or task.
• 

  liveBookmanning.com
  Grokking Machine LearningTransfer learning is a technique that adapts a pretrained model to a new task or dataset by continuing its training, often by modifying only some parameters.
• 

  liveBookmanning.com
  Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIA technique that adapts knowledge learned from one task or dataset to a related task.
• 

  liveBookmanning.com
  Sutskever's ListTransfer learning is a technique that adapts knowledge learned from one task or dataset to another task or dataset.
• 

  liveBookmanning.com
  Domain-Specific Small Language ModelsThe process of taking a model trained on one task or domain and applying it to a related task or domain, often with further tuning.
• 

  liveBookmanning.com
  Outlier Detection in PythonTransfer learning is a technique that adapts knowledge learned by a model on one task or dataset to another task or dataset.

Compare how 11 books explain transfer learning →

Related concepts

• fine-tuningused by · Fine-tuning is adapting a pretrained machine-learning model by training it further on targeted data or objectives for a task, domain, or behavior.
• pretrained modeluses · A pretrained model is a machine-learning model whose parameters were learned through prior training before it is used or adapted for a subsequent task.
• embeddinguses · A numerical vector representation of an item or input that encodes features or relationships useful for computational tasks.
• generalizationuses · Generalization is the extension of a pattern, model, solution, or conclusion beyond the specific examples or cases from which it was derived.
• unlabeled datauses · Unlabeled data consists of examples that lack target labels or task-specific annotations.
• causal invarianceused by · Causal invariance is the stability of a causal relationship across changes in environments or data distributions.

This explainer was generated by Manning's knowledge graph from the text of Knowledge Graphs and LLMs in Action. The section links point to the parts of the book it draws on.