Retrieval-augmented generation (RAG) supplies selected external information to a generative model at the time of a query. Rather than relying solely on the model’s learned parameters, it places relevant source material in the prompt, where it can inform the answer without changing those parameters. section 6.2.1 section 6.4.4
A large knowledge source may not fit in a model’s context window. RAG addresses this limit by selecting a small set of passages for each question instead of placing the entire source in the prompt. New documents can be indexed and retrieved without retraining the model. section 6.4.4 section 6.5
One retrieval pipeline works as follows:
- Convert the question into an embedding and compare it with document embeddings in an external knowledge base. section 6.4.4
- Rank the documents by similarity and insert the original text of high-ranking passages into the prompt alongside the question. section 6.4.4
- Generate an answer conditioned on the question, the retrieved passages, and the model’s learned parameters. section 6.4.4
Figure 6.8 Retrieval-augmented generation pipeline. A user query is transformed into an embedding, a vector representation used for similarity comparison against embedded documents in a knowledge base. The system ranks documents by semantic similarity and inserts only the top matches as text into the prompt before generation. The model’s response is thus conditioned on both its internal parameters and selectively retrieved external knowledge.

The pipeline distinguishes embeddings used for retrieval from the original text supplied to the model. section 6.4.4
Retrieval can reduce unsupported claims by bringing relevant evidence into view, but it does not guarantee a grounded answer. The retrieved material may be irrelevant or of poor quality; even relevant evidence can be underused, selectively interpreted, or extended with unsupported claims. Retrieved text remains subject to the model’s attention and positional limitations. section 6.2.1 section 6.4.5
Organizational knowledge
A corporate repository can be too large to fit in a single context window, making selective retrieval useful. In high-stakes deployments, retrieval alone is insufficient: data governance, traceability, access boundaries, integration, and regulatory compliance also matter. section 6.4.4 section 6.4.5













































