Transfer learning

As explained in Domain-Specific Small Language Models

The process of taking a model trained on one task or domain and applying it to a related task or domain, often with further tuning.

How Domain-Specific Small Language Models covers it

How deep it goes

Defines transfer learning and compares full, partial, and parameter-efficient tuning, without detailing implementation procedures.

Examples

Uses molecule descriptions and SMILES structures for generation and captioning.

What it assumes

Assumes a pretrained model and related target task, while defining how domain-specific data enables adaptation.

What you can do after

Apply domain-specific data to select a specialization strategy while weighing accuracy, computation, overfitting, and forgetting.

How it connects

Connects pretrained models with fine-tuning, parameter-efficient fine-tuning, catastrophic forgetting, and domain-specific LLMs.

liveBookmanning.com

Read Domain-Specific Small Language Models on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Inside Deep Learning: Math, Algorithms, ModelsA training approach that adapts a pretrained model to a new task, usually by reusing learned weights and matching the original preprocessing expectations.
• 

  liveBookmanning.com
  LLMs in ProductionTransfer learning is a machine-learning technique that adapts knowledge learned from one task or dataset to another task or domain.
• 

  liveBookmanning.com
  Natural Language Processing in Action, Second EditionTransfer learning is reusing knowledge learned from one task or dataset to improve learning on another task or dataset.
• 

  liveBookmanning.com
  Generative AI in ActionTransfer learning adapts knowledge learned from one task or dataset to improve performance on another task.
• 

  liveBookmanning.com
  Knowledge Graphs and LLMs in ActionTransfer learning is a machine-learning technique that reuses patterns learned from one task or domain to improve performance on another task or domain.
• 

  liveBookmanning.com
  Time Series Forecasting Using Foundation ModelsTransfer learning is a technique that applies knowledge learned by a model on one dataset or task to a different dataset or task.
• 

  liveBookmanning.com
  Grokking Machine LearningTransfer learning is a technique that adapts a pretrained model to a new task or dataset by continuing its training, often by modifying only some parameters.
• 

  liveBookmanning.com
  Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIA technique that adapts knowledge learned from one task or dataset to a related task.
• 

  liveBookmanning.com
  Sutskever's ListTransfer learning is a technique that adapts knowledge learned from one task or dataset to another task or dataset.
• 

  liveBookmanning.com
  Outlier Detection in PythonTransfer learning is a technique that adapts knowledge learned by a model on one task or dataset to another task or dataset.

Compare how 11 books explain transfer learning →

Related concepts

• fine-tuningused by · Fine-tuning is adapting a pretrained machine-learning model by training it further on targeted data or objectives for a task, domain, or behavior.
• pretrained modeluses · A pretrained model is a machine-learning model whose parameters were learned through prior training before it is used or adapted for a subsequent task.
• embeddinguses · A numerical vector representation of an item or input that encodes features or relationships useful for computational tasks.
• generalizationuses · Generalization is the extension of a pattern, model, solution, or conclusion beyond the specific examples or cases from which it was derived.
• unlabeled datauses · Unlabeled data consists of examples that lack target labels or task-specific annotations.
• causal invarianceused by · Causal invariance is the stability of a causal relationship across changes in environments or data distributions.

This explainer was generated by Manning's knowledge graph from the text of Domain-Specific Small Language Models. The section links point to the parts of the book it draws on.