Transfer learning reuses a model developed for one task as the starting point for a related task. Rather than learning everything from scratch, the new task benefits from knowledge the model has already acquired. Foundation models make this useful across language tasks: their broad representations can be adapted to work in new domains. section 2.1 section 9.1.1
Transfer learning describes the reuse of knowledge; fine-tuning is one way to adapt a pretrained model. Full fine-tuning updates all model parameters. Low-rank adaptation (LoRA) instead trains a smaller set of added components, reducing the number of parameters that need adjustment. The choice depends on the task, data, and model. section 9.1.1
Adapting across languages
A model trained on English text might be adapted for French text. The starting model supplies previously learned language knowledge, while adaptation targets the new language rather than beginning with an untrained model. section 9.1.1
For a fine-tuning project that uses transfer learning, follow the broader model-adaptation workflow:
- Choose a suitable pretrained model and adaptation method, considering the task, input and output sizes, dataset, and infrastructure. section 9.2.1
- Prepare representative task-specific data, including cleaning, normalization, and any required labeling. section 9.2.1
- Train and validate, then evaluate on test data before deployment. Check performance on the intended task and on new data, not just the training set. section 9.2.1
Transfer is most useful when the prior task’s knowledge is relevant to the new one; it does not remove the need for representative data or evaluation. For enterprise applications involving corporate or private data, account for privacy, security, and safety before production use. section 2.1 section 9.2.1 section 1.1
