liveBook · Manning

Transfer learning

As explained in Build AI Drug Discovery Pipelines

Transfer learning is a technique that adapts knowledge learned from one dataset or task to improve performance on another task.

Transfer learning adapts knowledge learned from one dataset or task to another. In drug discovery, a model pretrained on many unlabeled protein sequences or molecules can supply useful features for a task with limited labeled data. section 12.3.4 section 13.2.4

A pretrained protein language model such as ESM-2 learns general sequence features, including common motifs and structural tendencies. For a new classification task, add a classification head and fine-tune the model on labeled examples instead of training the entire classifier from scratch. section 12.3.4

Classifying antibody sequences

To distinguish HIV-1 from SARS-CoV-2 antibodies, start with a pretrained protein model, add a classification head, and fine-tune it on labeled antibody sequences. Compare results with a classifier trained from scratch; inspect the confusion matrix and performance across sequence lengths, not just a single score. section 12.3.4

Transfer can also use embeddings: numerical representations produced by a pretrained encoder. MegaMolBART embeddings encode small-molecule structure and can serve as inputs for downstream predictions such as solubility, binding affinity, or toxicity. The same encoder can support different tasks without designing separate molecular features for each one. section 13.2.4

Pretraining is especially useful when task-specific labels are scarce. For example, a variant-effect predictor fine-tuned from ESM-2 can use general protein patterns learned from millions of sequences, rather than learning those patterns solely from experimentally measured variants. section 13.2.4

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains classification-head fine-tuning and pretrained embeddings, but does not implement the transfer-learning workflow.
Examples
Illustrates antibody classification, protein variant-effect prediction, and small-molecule property prediction.
What you can do after
Evaluate/understand how pretrained representations can support downstream predictions and why they are useful with limited labels.
How it connects
Connects pretraining and fine-tuning with foundation models, embeddings, and classification.

Also explained in

Compare how 16 books explain transfer learning →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.