liveBook · Manning

Fine-tuning

As explained in Retrieval Augmented Generation, The Foundational Ideas

Fine-tuning is further training of a pretrained model on task-specific or domain-specific data.

A RAG system can retrieve the right passage and still answer from a half-remembered fact, or fill a gap that the passage leaves open. Fine-tuning addresses these behaviors by updating a pretrained model on task-specific examples. RAG supplies external evidence; fine-tuning changes how the system retrieves, ranks, and uses that evidence. They are complementary, not interchangeable. section 4.4 section 15.3

What gets updated depends on the architecture. REALM first jointly pretrains its retriever and reader on unlabeled text with a masked-language-modeling objective. It then fine-tunes both on a smaller labeled question-answering dataset, shifting the objective to predicting answer spans in retrieved documents. Canonical RAG instead starts with separately pretrained components. It fine-tunes the query encoder and generator while freezing the document encoder, so the existing document index does not need refreshing during training. section 3.2 section 4.1

For RAG question answering, training examples pair questions with target answers. Retrieved passages are included in the input while next-token loss is computed on the answer. The query encoder learns to fetch useful passages from the fixed index; the generator learns to answer from those passages and stop when they do not support an answer. This differs from merely placing retrieved text in a prompt and asking an off-the-shelf model to use it. section 4.4

Table 4.2 Canonical RAG vs naive RAG
Feature
Canonical RAG (Lewis et al., 2020)
Naive RAG (Industry Standard)
Definition
The original research involved sophisticated joint training and explicit probabilistic marginalization over retrieved documents.
A simplified, practical pattern using off-the-shelf components, relying on the LLM's inherent capabilities to integrate retrieved context.
Training
Jointly fine-tuned the retriever and generator for the specific task.
No joint training; uses a pretrained retriever and LLM.
Document Fusion
Employs explicit marginalization (e.g., RAG-Sequence, RAG-Token) to weigh document contributions.
Concatenates retrieved documents into the LLM's prompt; relies on the LLM's attention to handle context.
 
Complexity
Higher architectural and computational complexity requires specialized training.
Much simpler to implement and deploy; significantly lower computational cost.
Legacy
Established the fundamental retrieval-augmented generation pattern that subsequent work has refined but not replaced.
The default architecture for most real-world RAG applications is due to its accessibility and effectiveness.

Compare jointly fine-tuned canonical RAG with naive RAG, which uses pretrained components without joint training. The table makes the added training complexity—and the simpler deployment baseline—visible. section 4.7

Choose the fine-tuning target by the observed failure:

  • Missing domain-relevant results: fine-tune the embedding model on domain pairs to align retrieval with domain-specific relevance. section 15.3.3
  • Answers degrade as more passages are retrieved: train the reader with relevant and irrelevant passages. RAFT pairs an answer-bearing passage with distractors and an answer that quotes relevant text. section 15.3.3
  • Ranking and answering both need improvement: RankRAG instruction-tunes one LLM for context ranking and answer generation, rather than treating them as separate skills. section 15.3

When isolated passages are not enough

An unfine-tuned t5-small FiD test could keep passages separate and pick a strong answer signal, but struggled to weigh passages and synthesize an answer neither stated outright. Fine-tuning the decoder targets that evidence-combination skill; reported fine-tuned FiD results improved as the number of retrieved passages rose from 10 to 100. section 5.5

Start with the simpler baseline when it works: naive RAG needs no fine-tuning and less engineering. Fine-tune when evaluation identifies a retrieval, ranking, grounding, or noise-robustness failure worth the added training effort. LoRA or QLoRA adapters offer a lower-cost alternative to updating a full model. section 4.7 section 15.3.3

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Explains supervised objectives, which components update or stay frozen, and failure-driven training recipes; stops short of a full training implementation.
Examples
Uses REALM and canonical RAG training stages, a FiD synthesis failure, and RAFT examples mixing answer-bearing passages with distractors.
What it assumes
Assumes familiarity with pretrained models, retrieved passages, embeddings, and labeled question-answer pairs; explains which RAG components training updates.
What you can do after
Evaluate/understand when to retain naive RAG and when to target the retriever, reader, or generator with fine-tuning.
How it connects
Connects fine-tuning to retrieval-augmented generation, dense passage retrieval, question answering, context ranking, and adapter-based training.

Also explained in

Compare how 52 books explain fine-tuning →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.