liveBook · Manning

Retrieval-augmented generation

As explained in Grokking Software Architecture

Retrieval-augmented generation is an architecture that retrieves relevant external information and supplies it as context to a generative AI model before response generation.

Retrieval-augmented generation (RAG) connects a generative AI model to a database: retrieve relevant facts, add them to the user’s prompt, and direct the model to answer from that context. It can let an AI answer questions using private company documents rather than relying only on what it learned before. section 9.4.5 section 8.2.3

RAG addresses a basic mismatch: databases retrieve stored information, while AI models predict likely answers and may confidently invent facts. Retrieval helps, but it brings tradeoffs:

  • Vector search finds approximate nearest neighbors, not guaranteed exact truth; a close match can be dangerously wrong in medical or legal applications. section 8.2.3
  • Vector indexes can require substantially more memory and computing power than simple text indexes, making a dedicated vector database costly for some projects. section 8.2.3
Figure 9.3 AI models are probability engines, not databases. Without RAG guardrails, they will confidently improvise the wrong answer. RAG turns the AI from an "improv actor" into an "open-book test taker”.

The open-book-test analogy captures the idea: give the model relevant facts to use rather than trusting it to remember them. section 9.4.5

How Grokking Software Architecture covers it

How deep it goes
Explains retrieval and prompt augmentation, but does not specify indexing, retrieval tuning, or implementation steps.
Examples
Uses private-document question answering and an open-book-test analogy to illustrate the pattern.
What you can do after
Understand how retrieved facts can ground answers and recognize vector search’s accuracy and infrastructure tradeoffs.
How it connects
Connects language models with vector databases, semantic search, and private company documents.

Also explained in

Compare how 46 books explain retrieval-augmented generation →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Software Architecture. The section links point to the parts of the book it draws on.