A general-purpose LLM may write fluent answers yet mishandle medical terminology, company jargon, or a required output format. Fine-tuning adapts a pretrained model by training it further on task- or domain-specific examples. Unlike prompting or retrieval-augmented generation (RAG), it updates model weights, shaping how the model responds as well as what domain patterns it has learned. section 5.0 section 1.4.1
Choose the least costly intervention that meets the quality bar:
- Improve the prompt first and test it on real requests. If eight of ten responses meet the quality bar, stop. section 5.1.1
- Use RAG when answers depend on changing facts, such as current inventory. Fine-tuned knowledge remains static until the model is retrained. section 5.1.1
- Consider fine-tuning when stable domain conventions, tone, or formats remain inconsistent despite prompting. Combine it with RAG when the application also needs live knowledge. section 5.1.1 section 5.5
Table 5.1 Comparison of prompting, RAG, and fine-tuning as optimization strategies
|
Feature
|
Prompting
|
RAG
|
Fine-tuning
|
|---|---|---|---|
|
Weight updates
|
❌No
|
❌No
|
✅Yes
|
|
Domain knowledge
|
❌Limited
|
✅External sources
|
✅Baked into model
|
|
Cost and setup
|
 Cheapest
|
Medium (requires infrastructure)
|
Highest (requires data, compute, and up-front investment)
|
|
Custom tone/style
|
⚠️Hard to control
|
⚠️Limited control
|
✅Precise control
|
|
Knowledge freshness
|
⚠️Static
|
✅Dynamic
|
❌Static unless retrained
|
|
When to use
|
Quick experiments and basic tasks
|
Dynamic factual tasks and knowledge grounding (but subject to context window limits and retrieval latency)
|
Domain expertise and internal workflows
|
The comparison separates weight updates, knowledge freshness, setup cost, and style control. Fine-tuning offers stronger customization but requires data and compute; RAG supplies dynamic external knowledge. section 5.1
A practical workflow has three phases: collect, format, and validate examples; select a base model and fine-tuning method; then configure, run, and evaluate training. Training data can use prompt–completion pairs or structured chat conversations. Its quality and preparation largely determine the result. For retrieval systems, the same adaptation principle applies to embedding models: in-domain question–document pairs or synthetic questions can improve matching when generic embeddings miss specialized vocabulary. section 5.3 section 5.5 section 3.7.1
Customer-support assistant
Suppose a support bot must maintain a branded voice and handle difficult returns consistently. Put short system, user, and assistant exchanges into a JSONL training file. With the OpenAI Python SDK, upload the file for fine-tuning, create a job against a supported model, inspect its status and validation checkpoints, then call the resulting model for new conversations. An API-managed job handles training infrastructure; an open-source model trained with Hugging Face Transformers offers more control. LoRA or QLoRA can make large-model fine-tuning practical on modest hardware. section 5.3.2 section 5.5
Evaluate beyond the training task. Strong medical-exam results may not hold on novel clinical cases, and code-generation output still needs human review for complex logic. Audit the examples themselves: fine-tuning on support logs can reproduce unequal treatment present in human replies, and repeated training on those outputs can amplify the bias. section 5.2.1 section 5.2.3 section 11.2.1
