Fine-tuning continues training an existing model on a smaller, task-specific dataset. It adjusts the model’s internal parameters, producing an altered copy that behaves more consistently for a chosen purpose. section 3.3
A model learns a particular style
To fine-tune a language model, provide examples of text written in the intended style. The training algorithm uses those examples to refine the model; only a handful of documents may be needed compared with the data used for original training. section 3.3
Use fine-tuning as a deliberate escalation rather than the default customization method:
- Improve the prompt first, including few-shot examples or retrieved context through RAG. section 3.3 section 3.4
- Fine-tune when highly customized outputs remain necessary after prompt improvements, or when a smaller model is preferable for cost or self-hosting. section 3.4
- Watch for overfitting: excessive specialization can make the model memorize training examples and perform poorly on new instances. section 3.3
- Compare strategies with a separate validation set, then sanity-check the selected model with a test set; do not reuse the test set. section 3.15
Fine-tuning is especially practical with open source models because their parameters are accessible. It is not a shortcut around data quality, training configuration, or computing requirements; effective use may require machine-learning expertise and resources. section 3.3 section 3.4
Table 3.2 Off-the-shelf vs. fine-tuned AI
| Off-the-shelf AIBest when . . . |
Fine-tuned AIBest when . . . |
|---|---|
| • Prompt engineering techniques work well. • It is okay to use proprietary AI. • You can afford large models. • You prioritize ease of use. |
• You want highly customized outputs, and you’ve exhausted other options. • You need to use smaller models (for example, for self-hosting them). • You have ML expertise and access to computing resources. |
Compare when off-the-shelf AI is preferable with the conditions that justify fine-tuning. section 3.4
