Transfer learning adapts knowledge learned from one dataset or task to another. In drug discovery, a model pretrained on many unlabeled protein sequences or molecules can supply useful features for a task with limited labeled data. section 12.3.4 section 13.2.4
A pretrained protein language model such as ESM-2 learns general sequence features, including common motifs and structural tendencies. For a new classification task, add a classification head and fine-tune the model on labeled examples instead of training the entire classifier from scratch. section 12.3.4
Classifying antibody sequences
To distinguish HIV-1 from SARS-CoV-2 antibodies, start with a pretrained protein model, add a classification head, and fine-tune it on labeled antibody sequences. Compare results with a classifier trained from scratch; inspect the confusion matrix and performance across sequence lengths, not just a single score. section 12.3.4
Transfer can also use embeddings: numerical representations produced by a pretrained encoder. MegaMolBART embeddings encode small-molecule structure and can serve as inputs for downstream predictions such as solubility, binding affinity, or toxicity. The same encoder can support different tasks without designing separate molecular features for each one. section 13.2.4
Pretraining is especially useful when task-specific labels are scarce. For example, a variant-effect predictor fine-tuned from ESM-2 can use general protein patterns learned from millions of sequences, rather than learning those patterns solely from experimentally measured variants. section 13.2.4















