5 Fine-tuning LLMs for improved performance
This chapter covers
- Choosing the right approach for domain-specific tasks
- Preparing high-quality training data for fine-tuning
- Understanding the fine-tuning process for closed and open source models
- Building a reliable customer-support assistant
- Using knowledge distillation to create smaller, faster student models for production
“This model gives okay results, but it’s not precise enough for our medical terminology.”
“The AI generates good text, but it doesn’t understand our company’s internal jargon.”
“We need more accuracy for legal documents. General models make too many subtle errors.”
Do these concerns sound familiar? You’re not alone. Although today’s large language models (LLMs) demonstrate impressive capabilities across a wide range of tasks, they often fall short when facing highly specialized domains or complex workflows that require domain expertise.
In previous chapters, we discussed various techniques for optimizing LLM performance, including prompting (zero- and few-shot) and retrieval-augmented generation (RAG). These methods allow us to get better results from models without changing their underlying weights. In some cases, however, these techniques are insufficient, and more fundamental adjustments to the model’s behavior are necessary. This is where fine-tuning becomes valuable.
Figure 5.1 The fine-tuning process: training a pretrained model on specialized data to adapt its behavior for specific domains or tasks
