liveBook · Manning

Retrieval-augmented generation

As explained in Build an Advanced RAG Application (From Scratch)

Retrieval-augmented generation is a generation technique that retrieves relevant external information and supplies it to a language model when producing a response.

Retrieval-augmented generation (RAG) answers a query by retrieving relevant material from an external corpus, then giving that material to a language model as context for generation. It combines semantic retrieval with fluent response synthesis, supplying knowledge beyond a model’s training data. RAG is useful for focused, specialized, or current collections, but retrieval alone does not guarantee that the underlying information is reliable. section 1.2 section 3.0

Figure 6.1 Architecture of a Retrieval Augmented Generation (RAG) system showcasing the integration of retrieval and generative components for enhanced natural language processing tasks.

The architecture separates query processing, embedded knowledge, retrieval, prompt context, decoder generation, and the final response. section 6.1

A typical pipeline embeds documents or chunks, stores their vectors in a searchable index, embeds the incoming query, retrieves the closest passages, and inserts those passages into a prompt. The decoder then uses the retrieved facts, entities, and relationships to form a coherent answer. Grounding generation this way can reduce hallucinated or stale responses, while preserving conversational output. section 6.0 section 6.1

Implementation choices determine whether retrieval remains useful as content and traffic grow:

  • Chunk long documents at natural boundaries and retain overlap so embeddings fit context limits without losing continuity across chunks. section 6.4 section 6.6
  • Embed chunks and queries with the same semantic model, then retrieve nearest vectors with cosine-style similarity. section 6.1 section 6.4
  • Keep metadata with retrieved content so answers can cite or filter sources, such as hotel locality or paper year. section 6.4 section 6.6
  • Move from an experimental FAISS index to a persistent vector database such as Qdrant when concurrent access, reliability, and scale matter. section 6.4 section 6.6
  • Evaluate retrieval quality and generated-answer quality together; surface text-similarity metrics alone miss factual support and relevance. section 6.5 section 6.6

Hotel and research-paper assistants

A hotel assistant retrieves Paris reviews for a request such as “Hotels with a view of the Eiffel Tower,” then prompts a decoder to produce a cited recommendation. A research assistant recursively chunks paper abstracts, embeds and stores them in Qdrant, retrieves matching chunks, and generates an answer with paper metadata and citations. section 6.1 section 6.4

Basic RAG fits a single, predictable knowledge domain. Enterprise deployments add query rewriting, decomposition, intent-based routing, semantic caching, memory, and guardrails to avoid cross-domain noise, repeated computation, unsafe requests, and latency problems. These additions trade extra processing for improved precision, cost control, and operational behavior. section 7.0 section 7.5

How Build an Advanced RAG Application (From Scratch) covers it

How deep it goes
Builds complete retrieval-to-generation pipelines, including embeddings, chunking, FAISS or Qdrant storage, prompting, and joint evaluation.
Examples
Demonstrates Paris hotel recommendations from review retrieval and research-paper answers from chunked DBLP abstracts.
What it assumes
Builds on semantic search with encoder models and text generation with decoder models before combining them.
What you can do after
Build RAG assistants that retrieve semantic context, generate cited answers, and evaluate retrieval and answer quality.
How it connects
Connects semantic search, encoder models, decoder models, embeddings, vector databases, chunking, prompting, hallucination, and RAGAS.

Also explained in

Compare how 46 books explain retrieval-augmented generation →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build an Advanced RAG Application (From Scratch). The section links point to the parts of the book it draws on.