liveBook · Manning

Fine-tuning

As explained in LLM Customization and Fine-Tuning

Further training a pretrained model on task- or domain-specific data to change its behavior.

Fine-tuning further trains a pretrained model on curated examples to change its behavior. It can make a general model follow a domain’s conventions, produce a consistent output format, or adopt an enterprise voice. It is one option in an adaptation continuum that also includes prompting, retrieval-augmented generation (RAG), and preference optimization. section 1.11 section 6.5

The central decision is whether the gap is behavioral or knowledge-bound. If the model has the right information but responds in the wrong format or style, fine-tuning may help. If it needs current documents or facts it cannot reliably supply, use retrieval; many systems need both. Fine-tuning can memorize patterns in new facts without reliably generalizing across paraphrased questions. section 4.10 section 6.5

A costly mismatch

An investment bank fully fine-tuned an 8B model on historical earnings transcripts, spending eight weeks and $55,000 in compute. The model adopted financial language but could not answer questions about yesterday’s call. Retrieving current transcripts would have addressed the missing information. section 2.0

Choose and evaluate the technique in stages:

  • Start with a few-shot baseline. Add RAG for missing knowledge; try more examples when coverage is the problem. Consider fine-tuning when prompting plateaus or its per-request cost is too high. section 4.9.2
  • Reuse curated prompt examples as training-set seeds and keep a held-out set to compare base, prompted, and fine-tuned variants. Match the inference prompt format to the training format. section 4.9.2
  • Choose between low-rank adaptation (LoRA) and full supervised fine-tuning (SFT) with data, compute, and operational costs in mind. Put an evaluation gate before deployment and monitor for drift afterward. section 1.11 section 4.9.2
  • Train for the desired behavior explicitly. Generic instruction tuning can make wrong answers more confident, and adaptation can weaken safety properties. section 1.11

Economics depend on volume and assumptions. Few-shot prompting has low upfront cost but repeatedly sends examples as input tokens. A self-hosted fine-tuned model incurs training and hosting costs that must be spread across requests. Prompt caching, deployment urgency, latency, and engineering capacity can change the crossover. section 4.7.1 section 4.7.3

Table 4.6 ICL vs. fine-tuning crossover at four volumes (illustrative; units of X per million tokens)
Daily volume
Daily ICL spend
LoRA training one-off
LoRA daily hosting
Break-even
100 req/day
~0.2 X
~200 X
~10 X
Never (daily hosting alone exceeds API spend)
10K req/day
~20 X
~200 X
~10 X
About 3 weeks
100K req/day
~200 X
~200 X
~10 X
1 to 2 days
1M req/day
~2,000 X
~1,000 X (full SFT)
~100 X
Under 1 day

Compare illustrative prompting and fine-tuning costs at four request volumes; rerun the calculation with actual hosting and cached-token prices. section 4.7.3

Customer-service voice

SK Telecom fine-tuned GPT-4 on Korean telecom customer-service conversations. Against base GPT-4, it reported 35% better conversation summaries, 33% better intent recognition, and satisfaction rising from 3.6 to 4.5 out of 5. section 1.2.3

How LLM Customization and Fine-Tuning covers it

How deep it goes
Develops a decision path from prompting to fine-tuning, with evaluation, cost crossover, and deployment considerations.
Examples
Uses earnings transcripts, Korean customer-service conversations, and clinical question-answer pairs to illustrate adaptation choices.
What you can do after
Evaluate whether fine-tuning fits a behavior gap and compare it with prompting or retrieval using held-out examples and cost assumptions.
How it connects
Connects fine-tuning with few-shot prompting, retrieval-augmented generation, LoRA, supervised fine-tuning, and preference optimization.

Also explained in

Compare how 52 books explain fine-tuning →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Customization and Fine-Tuning. The section links point to the parts of the book it draws on.