A model-adaptation choice is not simply whether to fine-tune. The adaptation continuum ranges from changing prompts and supplying retrieved information to changing model weights, distilling behavior, and optimizing preferences. The practical question is which technique addresses the gap at an acceptable cost—and which to try first. section 1.1 section 2.1
Knowledge versus behavior
An investment bank spent eight weeks and $55,000 fine-tuning an 8B model on historical earnings transcripts. The model adopted financial language but could not answer questions about yesterday’s call. Retrieval-augmented generation (RAG) better fit the need to reason over current documents. section 2.0
Six recognizable options provide starting points, not a mandatory sequence:
- Prompting and in-context learning: test whether instructions or a few examples elicit the desired behavior without training. section 1.1 section 2.1
- RAG: retrieve documents when answers need changing knowledge or grounding in sources. section 2.1
- LoRA or QLoRA: use parameter-efficient updates for behavioral specialization, voice, or house style. section 2.1
- Full supervised fine-tuning (SFT): train the model more extensively on examples when the task warrants the added effort. section 1.1 section 2.1
- Distillation: transfer task behavior to a smaller deployable model, while checking refusal behavior separately. section 1.10 section 2.11
- Preference optimization: shape preferences such as tone or refusal as part of alignment. section 1.1 section 2.11
Table 2.1 The adaptation continuum at decision depth.
|
Technique
|
Data needed
|
Hardware
|
Cost
|
Timeline
|
Best for
|
|
Prompting and
in-context learning (chapter 4)
|
None to a
handful of examples, plus a held-out test set
|
None beyond
inference
|
Tokens only
|
Hours to days
|
First-pass
capability check; tasks frontier models already handle
|
|
RAG (chapter 4)
|
A corpus of
documents
|
Vector store
(software) plus inference
|
$50 to $500 per
month for the vector store, plus longer prompts on every call
|
Days to weeks
|
Knowledge that
changes; citation grounding; mixed-document reasoning
|
|
LoRA / QLoRA
(chapter 5)
|
60 to 500
examples
|
8 to 12 GB GPU
(QLoRA: 4 to 6 GB)
|
$300 to $2,000
|
Weeks
|
Behavioral
specialization, voice, and house style beyond what a response schema can
enforce
|
|
Full SFT
(chapter 6)
|
500 to 5,000
examples
|
About 32 GB of
GPU memory for a 4B model: one 40 to 80 GB GPU, or two 24 GB cards
|
$2,000 to
$20,000
|
4 to 8 weeks
|
Complex domain
reasoning when LoRA has plateaued
|
|
Distillation
(chapter 7)
|
Teacher API
access plus a few hundred to 10,000 examples
|
Frontier API
budget plus student GPU
|
$2,000 to
$10,000
|
6 to 12 weeks
|
High-volume
self-hosted deployments where you want frontier-model quality at
student-model cost
|
|
Preference
optimization (chapter 8)
|
500 to 5,000
preference pairs
|
24 GB GPU for
the adapter path, plus generation budget
|
$3,000 to
$15,000
|
4 to 8 weeks
|
Refusal
behavior, tone, hallucination reduction, safety alignment
|
Compare the options by data, hardware, cost, timeline, and fit; the figures are typical ranges, not guarantees. section 2.1
Start with a held-out baseline and consider prompting and RAG before a training run. The transition from RAG to LoRA introduces weight updates, training data, GPU resources, a training loop, and an evaluation gate. A heavier method is not automatically better: choose against the actual knowledge or behavior gap, as well as latency, safety, data-residency, and cost constraints. section 1.1 section 2.1 section 2.11
Compare build costs with recurring inference costs. Prompt examples and retrieved passages add tokens on every call, so cheap-to-build options can become expensive at volume. Techniques can also be combined: RAG supplies knowledge while LoRA or SFT specializes behavior. Plan for evaluation, monitoring, and rollback, and account for risks such as poisoned retrieval or weakened safety behavior. section 2.1 section 2.11
