liveBook · Manning

Adaptation continuum

As explained in LLM Customization and Fine-Tuning

A range of model-adaptation options, from prompting and retrieval to fine-tuning, considered as choices with different costs and complexity.

A model-adaptation choice is not simply whether to fine-tune. The adaptation continuum ranges from changing prompts and supplying retrieved information to changing model weights, distilling behavior, and optimizing preferences. The practical question is which technique addresses the gap at an acceptable cost—and which to try first. section 1.1 section 2.1

Knowledge versus behavior

An investment bank spent eight weeks and $55,000 fine-tuning an 8B model on historical earnings transcripts. The model adopted financial language but could not answer questions about yesterday’s call. Retrieval-augmented generation (RAG) better fit the need to reason over current documents. section 2.0

Six recognizable options provide starting points, not a mandatory sequence:

  • Prompting and in-context learning: test whether instructions or a few examples elicit the desired behavior without training. section 1.1 section 2.1
  • RAG: retrieve documents when answers need changing knowledge or grounding in sources. section 2.1
  • LoRA or QLoRA: use parameter-efficient updates for behavioral specialization, voice, or house style. section 2.1
  • Full supervised fine-tuning (SFT): train the model more extensively on examples when the task warrants the added effort. section 1.1 section 2.1
  • Distillation: transfer task behavior to a smaller deployable model, while checking refusal behavior separately. section 1.10 section 2.11
  • Preference optimization: shape preferences such as tone or refusal as part of alignment. section 1.1 section 2.11
Table 2.1 The adaptation continuum at decision depth.
Technique
Data needed
Hardware
Cost
Timeline
Best for
Prompting and in-context learning (chapter 4)
None to a handful of examples, plus a held-out test set
None beyond inference
Tokens only
Hours to days
First-pass capability check; tasks frontier models already handle
RAG (chapter 4)
A corpus of documents
Vector store (software) plus inference
$50 to $500 per month for the vector store, plus longer prompts on every call
Days to weeks
Knowledge that changes; citation grounding; mixed-document reasoning
LoRA / QLoRA (chapter 5)
60 to 500 examples
8 to 12 GB GPU (QLoRA: 4 to 6 GB)
$300 to $2,000
Weeks
Behavioral specialization, voice, and house style beyond what a response schema can enforce
Full SFT (chapter 6)
500 to 5,000 examples
About 32 GB of GPU memory for a 4B model: one 40 to 80 GB GPU, or two 24 GB cards
$2,000 to $20,000
4 to 8 weeks
Complex domain reasoning when LoRA has plateaued
Distillation (chapter 7)
Teacher API access plus a few hundred to 10,000 examples
Frontier API budget plus student GPU
$2,000 to $10,000
6 to 12 weeks
High-volume self-hosted deployments where you want frontier-model quality at student-model cost
Preference optimization (chapter 8)
500 to 5,000 preference pairs
24 GB GPU for the adapter path, plus generation budget
$3,000 to $15,000
4 to 8 weeks
Refusal behavior, tone, hallucination reduction, safety alignment

Compare the options by data, hardware, cost, timeline, and fit; the figures are typical ranges, not guarantees. section 2.1

Start with a held-out baseline and consider prompting and RAG before a training run. The transition from RAG to LoRA introduces weight updates, training data, GPU resources, a training loop, and an evaluation gate. A heavier method is not automatically better: choose against the actual knowledge or behavior gap, as well as latency, safety, data-residency, and cost constraints. section 1.1 section 2.1 section 2.11

Compare build costs with recurring inference costs. Prompt examples and retrieved passages add tokens on every call, so cheap-to-build options can become expensive at volume. Techniques can also be combined: RAG supplies knowledge while LoRA or SFT specializes behavior. Plan for evaluation, monitoring, and rollback, and account for risks such as poisoned retrieval or weakened safety behavior. section 2.1 section 2.11

How LLM Customization and Fine-Tuning covers it

How deep it goes
Compares six options by data, hardware, cost, and timeline, then develops techniques through end-to-end examples and tradeoffs.
Examples
Contrasts a bank’s costly fine-tuning mismatch with a ticket-classification task using eight examples and no training.
What you can do after
Evaluate adaptation options against task fit, resource requirements, and operational constraints, including when to combine techniques.
How it connects
Connects prompting, retrieval-augmented generation, parameter-efficient fine-tuning, supervised fine-tuning, distillation, and preference optimization.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Customization and Fine-Tuning. The section links point to the parts of the book it draws on.