liveBook · Manning

Chapter 11: Correction, planning, and reasoning

What you will learn in chapter 11 of Retrieval Augmented Generation, The Foundational Ideas

This introduction explains why better retrieval alone cannot fix single-pass RAG and contrasts two ways to add control: correcting retrieval afterward and planning it beforehand.

After this chapter you will be able to

What it builds on

Section by section

Introduction

This introduction explains why better retrieval alone cannot fix single-pass RAG and contrasts two ways to add control: correcting retrieval afterward and planning it beforehand.

Key terms: Retrieve-then-Generate, single-pass, control flow, CRAG, correction-after, Retrieval Evaluator, fallback, decompose-then-recompose, LevelRAG, plan-before

11.1 The agentic turn

This section defines the shift from single-pass RAG to agentic RAG and identifies recurring retrieval and answer failures that motivate it.

Key terms: Agentic RAG, Single-Pass RAG, control flow, cyclical feedback loop, validation step, FP1: Missing Content or Staleness, FP2: Missed the Top Rank, FP6: Incorrect Specificity, FP7: Incomplete Answers

11.1.1 The enterprise compliance assistant

The section introduces a bank compliance scenario to show why a high-stakes onboarding question requires checking several potentially conflicting sources rather than relying on a single-pass RAG answer.

Key terms: Enterprise Compliance Assistant, Internal Policy Documents, External Regulatory Directives, Sanctions Lists, Client Profiles, politically exposed person (PEP), expedited KYC, Single-Pass RAG, approval matrices

11.1.2 Anatomy of a single-pass failure

This section uses a retrieval trace to show how a single-pass system can favor a similar but invalid policy, then identifies the checks that could catch the error.

Key terms: Single-Pass system, vector similarity score, validity, metadata dates, EU Directive, Internal Policy, CRAG evaluator, Ambiguous, LevelRAG planner, EU PEP Rules 2024

11.1.3 Stale knowledge and missing freshness checks

This section explains why Single-Pass RAG can return outdated regulatory evidence and what a freshness check and live-search fallback would add.

Key terms: Single-Pass RAG, snapshot of knowledge, staleness, FP1: Missing Content or Staleness, evaluator, potentially stale, live web search, cached evidence

11.1.4 Retrieval collapse on long-tail entities

This section explains how retrieval can favor a famous namesake over an obscure entity and why that failure is especially serious in compliance work.

Key terms: long-tail entities, dense retriever, Retrieval Collapse, Know Your Customer (KYC), FP2: Missed the Top Rank, FP6: Incorrect Specificity

11.1.5 Multi-hop reasoning gaps

This section explains why comparative questions can defeat single-pass RAG and how adaptive retrieval addresses the gaps left by fixed query decomposition.

Key terms: multi-hop reasoning, comparative, UK threshold, EU threshold, Single-Pass RAG, fragmented context, FP7: Incomplete Answers, RAG-Fusion, multi-query retrieval, HyDE

11.2 Corrective retrieval augmented generation

The section introduces CRAG as a lightweight way for a RAG system to check retrieved documents and seek better sources before generating an answer.

Key terms: Corrective Retrieval Augmented Generation (CRAG), Trust, but Verify, quality check, fallback, evaluation node, agentic spectrum, index layer

11.2.1 The retrieval evaluator

This section explains how CRAG's Retrieval Evaluator scores retrieved evidence, routes the system to corrective actions, and remains limited by documents the retriever failed to return.

Key terms: Retrieval Evaluator, discriminative task, scalar confidence score, generative LLM, Correct, Incorrect, Ambiguous, Knowledge Refinement, Web Search, Hybrid strategy

11.2.2 Decompose-then-recompose

This section explains how CRAG turns a retrieved document into a shorter, more relevant context for the generator.

Key terms: Decompose-then-Recompose, knowledge strips, Evaluator, recomposition, Lost-in-the-Middle errors, signal-to-noise ratio, boilerplate

11.2.3 Corrective actions and web search integration

This section explains how CRAG uses web search when internal retrieval is inadequate and combines external findings with internal context.

Key terms: Incorrect, Ambiguous, web search fallback, query rewriter, Decompose-then-Recompose, internal context, external finding, OFAC

11.2.4 The CRAG pipeline

This section implements a branching CRAG pipeline for a compliance query and shows how evaluator and threshold choices can either catch stale evidence or filter out the correction.

Key terms: RetrievalEvaluator, CRAGPipeline, branching control flow, quality gate, ambiguity zone, Incorrect, Ambiguous, web_searcher, Decompose-then-Recompose, cross-encoder

11.2.5 Evaluation, shortcomings, and where CRAG fits

This section evaluates CRAG’s benefits and costs, shows why external evidence checks remain useful, and defines the limits of its defensive-retrieval pattern.

Key terms: CRAG, web search fallback, retrieval noise, latency trade-offs, Self-CRAG, external evaluator, reflection tokens, Search-o1, Reason-in-Documents, agentic spectrum

11.3 LevelRAG

Introduces LevelRAG as a hierarchical way to plan and verify retrieval, and distinguishes its approach from CRAG’s correction of retrieved results.

Key terms: High-Level Searcher, Low-Level Searchers, atomic sub-queries, adaptive routing, feedback loop, verify, supplemental queries, CRAG, LevelRAG, IRCoT

11.3.1 The high-level searcher

The section explains how LevelRAG’s High-Level Searcher plans research, condenses retrieved evidence, and checks whether it is sufficient to answer a complex question.

Key terms: High-Level Searcher, HLS, Multi-hop Logic Planning, Query Decomposition, atomic queries, per-query summarization, atomic query, context window, Lost-in-the-Middle, Verify

11.3.2 The low-level searcher

This section explains how LevelRAG combines specialized searchers and uses retrieval results to decide what to search next.

Key terms: Low-Level Searchers, Sparse searcher, Dense searcher, Web searcher, Lucene syntax operators, Extend, Filter, Emphasize, Pseudo-Document Generation, atomic query

11.3.3 LevelRAG implementation

This section implements LevelRAG’s plan-and-verify retrieval loop, then uses a compliance example and evaluation findings to show its benefits, limits, and operational costs.

Key terms: HighLevelSearcher, LevelRAGEngine, atomic query, verify, supplement, Low-Level Searchers, single-searcher routing, retrieval modality, decomposition, INSUFFICIENT

11.4 Reactive versus proactive agency

The section contrasts CRAG’s reactive correction with LevelRAG’s proactive planning, compares when each is useful, and shows how they can work together.

Key terms: CRAG, LevelRAG, reactive, proactive, atomic sub-queries, verify step, staleness, multi-hop, latency, integration

11.5 Agentic RAG in practice

Introduces a broader view of agentic RAG, previewing how CRAG and LevelRAG relate to deployed systems.

11.5.1 The agentic spectrum

This section places agentic RAG approaches on a spectrum of control flow and explains when their added cost is justified.

Key terms: lightly agentic, plan-then-execute, plan-evaluate-replan, tool-using research agents, CRAG, LevelRAG, single pass, latency, tokens, regulated industry

11.5.2 Products and platforms

This subsection connects agentic RAG ideas to products and frameworks that support planning, evidence checking, and citation grounding.

Key terms: agentic RAG, query decomposition, retrieval strategies, evidence audits, citation grounding, LangGraph, Haystack, LlamaIndex Workflows, agent primitives, ReAct (Reason-Act)

11.5.3 How CRAG and LevelRAG fit together in production

Shows how a production pipeline combines LevelRAG planning with CRAG correction, then limits its compute cost by routing simple queries elsewhere.

Key terms: HighLevelSearcher, Low-Level Searcher, atomic sub-queries, CRAG-style evaluator, verify step, query complexity, classifier, agentic pipeline, single-pass path

11.5.4 Refusal as a feature

The section presents refusal based on retrieval confidence as a feature of corrective RAG and contrasts one-shot question answering with planned research by agents.

Key terms: Retrieval Validation, Confidence Score, Refusal as a Feature, LevelRAG, Query Planning, one-shot QA bots, Research Agents

11.6 Summary

Where this is used later

Concepts explained in this chapter

← Chapter 10: RAPTOR: Recursive abstractive processing for tree-organized retrieval Chapter 12: Graph-based RAG →

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.