liveBook · Manning

Retrieval-augmented generation

As explained in LLM Customization and Fine-Tuning

A technique that retrieves relevant external information and supplies it to a generative model when producing a response.

A financial assistant fine-tuned on historical earnings transcripts can learn financial language yet still miss what happened on yesterday’s call. Retrieval-augmented generation (RAG) addresses that knowledge gap by supplying relevant documents when the model answers, without changing its weights. section 2.0 section 4.5

A RAG pipeline indexes documents, retrieves relevant chunks for a query, and places them alongside the question in a prompt. The model conditions its response on that context. Updating the corpus can make new information available without retraining; retrieval quality limits what the model can use. section 1.6.2 section 4.5

Start with a measured prompting baseline. Add RAG when errors point to missing or outdated facts, particularly when information changes frequently or answers need verifiable citations. Fine-tuning addresses a different problem: behavior such as applying conventions consistently. A production system can combine retrieval for knowledge with an adapter for behavior. section 1.7 section 1.6.2 section 4.5

Contract review

For a legal review system, retrieve current regulatory text at inference time and use fine-tuning to shape structured tables and the organization’s risk-rating taxonomy. The retrieved documents can change without repeating the behavior training. section 6.5

Evaluate and debug the pipeline in layers:

  • Measure retrieval hit rate, answer faithfulness, and end-to-end accuracy separately; each points to a different part of the system. section 4.10
  • If the right chunk is retrieved but ignored, instruct the model to answer only from context and admit when context is insufficient. Test whether it follows that instruction using deliberately wrong context. section 4.5.6
  • If the wrong chunk is returned, inspect production failures: determine whether the answer was dropped or split during chunking, or whether an indexed answer was ranked too low. Revisit chunking, hybrid search, or reranking accordingly. section 4.5.6
  • Guard against invented answers on retrieval gaps with an explicit uncertainty instruction and a downstream check. Monitor retrieval, prompt assembly, and inference latency, and cap retrieved context length. section 4.5.6

A minimal pipeline can use vector embeddings and NumPy retrieval. Production extensions include chunking strategies, reranking, query rewriting, and hybrid retrieval. Choose additions based on measured failures rather than assuming a larger pipeline will fix every gap. section 4.10 section 4.5.6

How LLM Customization and Fine-Tuning covers it

How deep it goes
Works through a minimal retrieval pipeline, separate evaluation measures, production extensions, and failure diagnostics.
Examples
Uses earnings-call questions and legal contract review to distinguish fresh knowledge from learned behavior.
What you can do after
Build a minimal retrieval pipeline and diagnose retrieval, grounding, and latency failures.
How it connects
Connects RAG to in-context learning, few-shot prompting, fine-tuning, and low-rank adaptation.

Also explained in

Compare how 46 books explain retrieval-augmented generation →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Customization and Fine-Tuning. The section links point to the parts of the book it draws on.