liveBook · Manning

What you will learn in Retrieval Augmented Generation, The Foundational Ideas

Principles for architecting reliable and verifiable AI

by Ben Auffarth · Manning Early Access Program (MEAP)

Chapter by chapter

Chapter 1: How RAG research prevents disasters

The introduction defines RAG, shows why access to a correct source does not guarantee a grounded answer, and frames reliability as an engineering problem.

1.1 How this book teaches RAG · 1.2 How do RAG systems work? · 1.3 How RAG research addresses failure points · 1.3.1 The knowledge boundary problem · 1.3.2 The hallucination challenge · 1.3.3 The private knowledge access problem · 1.3.4 RAG is an enduring architectural pattern · 1.3.5 Why RAG justifies infrastructure investment · 1.4 Failure points within RAG systems · 1.4.1 RAG failure examples · 1.4.2 Research-backed solutions · 1.5 Naive, advanced, modular, and agentic RAG · 1.5.1 Naive RAG: Establishing the foundation · 1.5.2 Advanced RAG: Optimizing the components · 1.5.3 Modular RAG: Adaptive and composable systems · 1.5.4 Agentic RAG: Autonomous information-seeking systems · 1.5.5 RAG evolution · 1.6 Summary

What you learn in chapter 1 →

Chapter 2: Revolutions in semantics, scale, and similarity

The chapter introduction explains how advances in semantic representation, scalable vector search, and sentence similarity converged to enable RAG, while identifying retrieval trade-offs that still matter.

2.1 Word2Vec (2013): Semantic word embeddings · 2.1.1 From search engines to semantic understanding · 2.1.2 Word2Vec: From words to vectors · 2.1.3 Implementing Word2Vec: From theory to practice · 2.1.4 Business impact · 2.1.5 The foundation for dense retrieval · 2.1.6 Limitations and the path forward · 2.2 FAISS (2017): Billion-scale similarity search · 2.2.1 Approximate nearest neighbors · 2.2.2 The scale advantage · 2.2.3 The bridge to practical semantic search · 2.3 Sentence-BERT (2019): Practical sentence similarity · 2.3.1 From cross-encoders to bi-encoders · 2.3.2 Implementation and business impact · 2.3.3 Training, performance, and limitations · 2.3.4 The final piece of the RAG puzzle · 2.4 Ready for revolution · 2.4.1 The modern retrieval stack · 2.4.2 From retrieval to synthesis · 2.5 Summary

What you learn in chapter 2 →

Chapter 3: REALM: Birth of end-to-end trainable RAG

The introduction explains why retrieving similar documents is not enough to answer questions and presents REALM’s central shift toward training retrieval for usefulness to the reader.

3.1 Open-domain question answering · 3.2 Retrieval-augmented language model pre-training · 3.2.1 Test results · 3.3 Practical implementation with Python · 3.4 REALM's enduring impact and legacy · 3.5 Summary

What you learn in chapter 3 →

Chapter 4: Retrieval-augmented generation for knowledge tasks

Introduces canonical RAG’s modular design, its evidence-weighting distinction from naive RAG, and the practical advantages that helped it spread.

4.1 The RAG model architecture · 4.1.1 The retriever · 4.1.2 The generator · 4.1.3 Modular architecture · 4.2 Marginalization over retrieved documents · 4.2.1 How marginalization works in practice · 4.2.2 The importance of marginalization · 4.3 RAG-sequence vs. RAG-token variations · 4.3.1 RAG-sequence · 4.3.2 RAG-token · 4.3.3 RAG-sequence vs. RAG-token · 4.4 Fine-tuning for downstream tasks · 4.5 Case study: Open-domain question answering · 4.5.1 Round 1: Exact answers · 4.5.2 Round 2: Efficiency · 4.5.3 Round 3: Knowledge update · 4.5.4 Ablation studies: Why each component matters · 4.5.5 Where RAG struggled · 4.5.6 The verdict · 4.6 Building a basic RAG pipeline · 4.7 Canonical RAG vs. naive RAG · 4.7.1 The RAG ecosystem · 4.7.2 Limitations of naive RAG · 4.8 Summary

What you learn in chapter 4 →

Chapter 5: Fusion-in-Decoder for multi-document processing

The introduction explains why answers spread across retrieved documents are difficult for conventional RAG and presents FiD’s separation of independent encoding and fused decoding as a scalable way to synthesize them.

5.1 The challenges of multiple document contexts · 5.2 Independent encoding, fused decoding · 5.3 Scaling to hundreds of documents · 5.3.1 Scaling training · 5.4 Implementing FiD with transformer models · 5.5 Case study: Finding the FiD sweet spot · 5.6 Applications and performance characteristics · 5.6.1 FiD's evolution · 5.7 Summary

What you learn in chapter 5 →

Chapter 6: Atlas: Few-shot learning with retrieval augmentation

Introduces Atlas as a way to achieve strong few-shot performance by aligning a retriever and reader around external knowledge rather than relying on ever-larger models.

6.1 The parametric vs. non-parametric knowledge tradeoff · 6.2 Atlas pre-training methodology · 6.2.1 Joint training without labels · 6.2.2 The training objectives · 6.2.3 Code in practice · 6.3 Dynamic knowledge retrieval · 6.3.1 Head-to-head · 6.3.2 The temporal sensitivity experiment · 6.4 Efficient knowledge base updating · 6.4.1 Three strategies for efficient updates · 6.4.2 Compressing the index · 6.5 Few-shot learning strategies for RAG systems · 6.6 Joint training after Atlas · 6.6.1 RA-DIT: The same two components, tuned separately · 6.6.2 When you do not own the reader · 6.7 Case study: Enterprise knowledge adaptation · 6.8 Summary

What you learn in chapter 6 →

Chapter 7: HyDE: Imagining the answer before you search

Introduces query-document mismatch and HyDE’s idea of searching with an imagined answer while grounding the final response in retrieved documents.

7.1 The query-document mismatch problem · 7.2 Generating hypothetical documents · 7.3 Zero-shot dense retrieval without relevance labels · 7.4 Implementing HyDE for challenging queries · 7.5 Case study: Medical knowledge retrieval · 7.6 Variations and extensions · 7.6.1 Multi-sample HyDE · 7.6.2 Hybrid HyDE (Fusion) · 7.6.3 HyDE with re-ranking · 7.6.4 Reverse HyDE · 7.6.5 Learned query refinement: RQ-RAG · 7.7 Summary

What you learn in chapter 7 →

Chapter 8: RAG-Fusion: Multi-query retrieval enhancement

Introduces the retrieval coverage problem and explains how RAG-Fusion uses varied queries and rank-based result merging to give an answer generator broader context.

8.1 The coverage problem · 8.2 LLM-driven query generation · 8.2.1 Generating queries with LangChain · 8.3 The Reciprocal Rank Fusion algorithm · 8.4 Implementing multi-query RAG systems · 8.5 Case study: Enhancing e-commerce product search · 8.6 Balancing diversity and precision · 8.7 Summary

What you learn in chapter 8 →

Chapter 9: Self-RAG: Retrieval with reflection and self-critique

This introduction explains why Self-RAG makes retrieval and answer checking adaptive, how reflection tokens make those decisions visible, and how its original training approach differs from common practice today.

9.1 The challenges of controlling retrieval and generation · 9.1.1 The limitations of passive retrieval architectures · 9.1.2 The evolution toward self-reflection and adaptive retrieval · 9.2 Reflection tokens for retrieval, relevance, and critique · 9.2.1 The concept of self-reflection · 9.2.2 The reflection token taxonomy · 9.2.3 Reflection tokens as a monitoring signal · 9.3 Training for self-evaluation · 9.3.1 The critic-driven training pipeline · 9.3.2 Inference: Tree decoding and control · 9.3.3 Adaptive retrieval thresholding · 9.4 Implementing prompted Self-RAG · 9.4.1 Defining the graph state · 9.4.2 Emulating reflection tokens · 9.4.3 The control logic · 9.5 Real-world applications · 9.5.1 What the tokens cannot see · 9.6 Where Self-RAG fits, and what replaced it · 9.7 Related research: Fact-checking on top of RAG · 9.8 Summary

What you learn in chapter 9 →

Chapter 10: RAPTOR: Recursive abstractive processing for tree-organized retrieval

The introduction explains why flat retrieval loses context and how RAPTOR’s tree of chunks and summaries supports retrieval at different levels of detail.

10.1 The semantic flattening problem · 10.1.1 The limits of fixed-size chunking · 10.2 From lists to trees · 10.2.1 The recursive mechanism · 10.2.2 Visualizing the tree construction · 10.3 Clustering in high dimensions · 10.3.1 The curse of dimensionality and UMAP · 10.3.2 Gaussian Mixture Models and soft clustering · 10.3.3 Picking the number of clusters with Bayesian Information Criterion (BIC) · 10.4 Practical implementation · 10.4.1 The RaptorNode architecture · 10.4.2 The clustering engine · 10.4.3 The summarization engine · 10.4.4 The recursive tree builder · 10.5 Indexing and retrieval strategies · 10.5.1 Collapsed tree (flattened index) · 10.5.2 Implementing tree traversal · 10.5.3 When clustering across documents is the wrong idea · 10.6 Advanced summarization strategies · 10.7 Evaluation, performance, and legacy · 10.7.1 Benchmark performance · 10.7.2 Operational challenges · 10.7.3 Solutions and successors: Related architectures · 10.8 Summary

What you learn in chapter 10 →

Chapter 11: Correction, planning, and reasoning

This introduction explains why better retrieval alone cannot fix single-pass RAG and contrasts two ways to add control: correcting retrieval afterward and planning it beforehand.

11.1 The agentic turn · 11.1.1 The enterprise compliance assistant · 11.1.2 Anatomy of a single-pass failure · 11.1.3 Stale knowledge and missing freshness checks · 11.1.4 Retrieval collapse on long-tail entities · 11.1.5 Multi-hop reasoning gaps · 11.2 Corrective retrieval augmented generation · 11.2.1 The retrieval evaluator · 11.2.2 Decompose-then-recompose · 11.2.3 Corrective actions and web search integration · 11.2.4 The CRAG pipeline · 11.2.5 Evaluation, shortcomings, and where CRAG fits · 11.3 LevelRAG · 11.3.1 The high-level searcher · 11.3.2 The low-level searcher · 11.3.3 LevelRAG implementation · 11.4 Reactive versus proactive agency · 11.5 Agentic RAG in practice · 11.5.1 The agentic spectrum · 11.5.2 Products and platforms · 11.5.3 How CRAG and LevelRAG fit together in production · 11.5.4 Refusal as a feature · 11.6 Summary

What you learn in chapter 11 →

Chapter 12: Graph-based RAG

This introduction explains when relationship-heavy questions call for graph-based retrieval and distinguishes the roles a graph can play in RAG.

12.1 Graph-based RAG · 12.1.1 The core architecture of graph-based RAG · 12.2 Microsoft GraphRAG · 12.2.1 G-Indexing: Building the pyramid · 12.2.2 G-Retrieval and G-Generation: Map-reduce over the pyramid · 12.2.3 Implementing GraphRAG · 12.3 HippoRAG · 12.3.1 From PageRank to Personalized PageRank · 12.3.2 How HippoRAG works · 12.3.3 G-Generation: ranking by activation · 12.3.4 Implementing PPR activation · 12.4 LightRAG · 12.4.1 G-Indexing: Graph-enhanced indexing · 12.4.2 G-Retrieval: Dual-path routing · 12.4.3 Implementing LightRAG · 12.4.4 The three architectures on one question · 12.5 Evaluation and real-world application · 12.6 Emerging optimizations · 12.6.1 TagRAG · 12.6.2 Clue-RAG · 12.6.3 HybGRAG · 12.6.4 AcademicRAG · 12.6.5 Where this is going · 12.7 Summary

What you learn in chapter 12 →

Chapter 13: Context compression and pruning

This introduction explains why long retrieved context can hurt RAG answers and outlines ways to reduce or prioritize what the generator processes.

13.1 The costs of long context · 13.1.1 Evolution of context compression · 13.1.2 Types of compression · 13.2 Perplexity-based compression with LongLLMLingua · 13.2.1 Why absolute perplexity fails · 13.2.2 Contrastive perplexity as a relevance signal · 13.2.3 The LongLLMLingua pipeline · 13.2.4 Implementing a prompt compressor with the LLMLingua · 13.3 Native pruning with AttentionRAG · 13.3.1 Beyond all-layer aggregation: Evaluator heads · 13.4 Extensions: The RECOMP framework · 13.4.1 CORE and multi-hop synthesis · 13.5 Implementing a query-aware compressor · 13.5.1 Recursive retrieval and index nodes · 13.6 Case study: Legal document synthesis · 13.7 Tradeoffs: Compression ratio versus semantic loss · 13.8 Future directions and adjacent research · 13.8.1 Gist tokens and trained prompt compression · 13.8.2 KV-cache eviction · 13.8.3 Multi-modal retrieval · 13.8.4 Hardware-aware budgeting · 13.8.5 Agentic self-compression · 13.9 Summary

What you learn in chapter 13 →

Chapter 14: Evaluating RAG systems

The introduction shows why RAG evaluation needs separate retrieval and answer measures, while cautioning that LLM-judge scores are most useful for diagnosing and comparing systems rather than certifying quality.

14.1 Classical retrieval metrics and their limits · 14.1.1 The foundation: Precision and recall · 14.1.2 The correlation gap: When good retrieval metrics produce bad answers · 14.2 The RAGAS evaluation framework · 14.2.1 Faithfulness: does the answer stick to the evidence? · 14.2.2 Answer relevance: Does the answer address the question? · 14.2.3 Context relevance: Did retrieval fetch what was needed? · 14.2.4 Putting the triad to work · 14.2.5 Limitations and caveats · 14.2.6 Beyond the triad: When ground-truth still earns its keep · 14.3 LLM-as-judge: From G-Eval to trained evaluators · 14.3.1 G-Eval: Chain-of-thought meets evaluation · 14.3.2 ARES: Trained judges with statistical guarantees · 14.3.3 The bias problem: What LLM judges get wrong · 14.4 Building your evaluation dataset · 14.4.1 Generating balanced evaluation data from your corpus · 14.4.2 Tools that do this for you · 14.5 Implementing an evaluation pipeline · 14.5.1 A faithfulness judge · 14.5.2 Mapping scores to failure points · 14.5.3 Practical considerations at scale · 14.5.4 Telling a real change from noise · 14.5.5 Evaluating agents and conversations · 14.6 Case study: Continuous evaluation for a compliance assistant · 14.7 Summary

What you learn in chapter 14 →

Chapter 15: Production RAG: Metrics, agentic systems, and continuous improvement

The introduction argues that production RAG performance depends heavily on how well the reader handles noisy retrieval, and distinguishes two ways readers respond to more context.

15.1 The evolving RAG landscape · 15.1.1 More retrieval is not always better · 15.1.2 Measuring stability and scalability · 15.1.3 The reader is the bottleneck · 15.2 Agentic RAG: Autonomous retrieval systems · 15.2.1 Four design patterns · 15.2.2 From patterns to pipelines · 15.2.3 Practical implementation · 15.2.4 When agentic RAG helps and when it does not · 15.3 Fine-tuning in the RAG lifecycle · 15.3.1 RankRAG: Unifying ranking and generation · 15.3.2 Results and implications · 15.3.3 Where fine-tuning fits in the RAG lifecycle · 15.4 Self-improving RAG: The feedback loop · 15.4.1 The gatekeeper: Learning when to trust retrieval · 15.4.2 The critic: Learning which memories to trust · 15.4.3 The student: Learning to teach itself to use retrieval · 15.4.4 Building the feedback loop · 15.5 Practical frameworks for RAG technique selection · 15.5.1 Complexity versus performance · 15.5.2 Mapping failure modes to RAG approaches · 15.5.3 Decision tree for new projects · 15.6 Best practices for production deployment · 15.6.1 Instrument everything · 15.6.2 Version your pipeline as a unit · 15.6.3 RAG framework comparison · 15.6.4 The testing pyramid for RAG · 15.6.5 When the page is a picture · 15.7 Case study: Agentic customer support in fintech · 15.7.1 The problem · 15.7.2 The agentic design · 15.7.3 Results · 15.7.4 What worked and what did not · 15.7.5 Connecting to the failure points · 15.7.6 Lessons for production teams · 15.8 Current challenges and open research questions · 15.8.1 Evaluation remains fragmented · 15.8.2 Robustness and security · 15.8.3 The efficiency-quality tradeoff · 15.8.4 Retrieval across heterogeneous sources · 15.8.5 Open questions · 15.9 Summary

What you learn in chapter 15 →

Concepts explained in this book

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.