liveBook · Manning

Iterative retrieval

As explained in Retrieval Augmented Generation, The Foundational Ideas

Iterative retrieval is a retrieval strategy that performs multiple retrieval steps, using intermediate results or reasoning to formulate subsequent searches.

Iterative retrieval improves on a fixed retrieve-and-generate pipeline by using intermediate results, relevance judgments, or reasoning to decide what to search next. It addresses cases where one retrieval pass misses evidence, returns ambiguous context, or cannot assemble an answer spanning multiple sources. section 15.2 section 4.3.3

Common forms differ in how they use intermediate retrieval results:

  • Recursive retrieval follows a hierarchy: retrieve a compressed parent summary, assess its relevance, then follow its pointer to specific uncompressed child nodes. section 13.5.1
  • Agentic retrieval lets an agent decide whether to retrieve, reformulate a failed query, generate sub-queries, or continue searching across multiple sources. section 15.2
  • Multi-step retrieval is most appropriate when queries require multi-hop reasoning; simple factual lookups can begin with naive RAG and reranking. section 15.5.3

Searching a codebase

An AI coding assistant can search for a symbol, read the most relevant files, pull in additional context when the first results fall short, and reformulate the search when nothing useful appears. This loop replaces a predetermined sequence with retrieval decisions made at runtime. section 15.2

Table 15.1 RAG techniques by complexity and performance impact
Technique
Complexity
Primary failure points addressed
When to use
Naive RAG (ch. 4)
Low
Baseline
Simple factual queries over clean corpora
Hybrid search (ch. 2, 8)
Low
FP1, FP2
Queries mixing keywords and concepts
Reranking (ch. 8)
Low-Medium
FP2, FP4
Retrieval noise degrades generation
HyDE (ch. 7)
Medium
FP1, FP2
Abstract or underspecified queries
RAG-Fusion (ch. 8)
Medium
FP1, FP2, FP7
Diverse query interpretations needed
RAPTOR (ch. 10)
Medium
FP5, FP6
Large documents; queries at varying abstraction levels
Context compression (ch. 13)
Medium
FP4, FP5
Long or noisy retrieved contexts
Self-RAG (ch. 9)
High
FP3, FP5, FP7
Hallucination-sensitive applications
FLARE (not implemented; see the ch. 11 planner)
High
FP1, FP7
Long-form answers; information needs emerge mid-generation
CRAG (ch. 11)
High
FP1, FP3
Retrieval quality varies by query
LevelRAG (ch. 11)
High
FP1, FP7
Multi-part queries needing planned decomposition
Graph RAG (ch. 12)
High
FP1, FP7
Multi-hop questions over connected entities
Adaptive-RAG (ch. 15)
High
All
Mixed query complexity
Agentic RAG (ch. 15)
Very High
All
Complex multi-step queries, fragmented knowledge

Use the complexity table to compare iterative retrieval with lower-complexity techniques and identify when added latency, cost, and failure modes are justified. section 15.5.1

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Explains fixed versus autonomous pipelines and recursive hierarchy traversal, including runtime decisions but not a full implementation.
Examples
Uses hierarchical financial reports, compressed parent summaries, codebase assistants, reformulated searches, and generated sub-queries as examples.
What it assumes
Assumes a working RAG pipeline and evaluation baseline, while treating simple retrieve-and-generate systems as the starting comparison.
What you can do after
Apply it via agentic RAG or recursive retrieval when evaluation shows that a single retrieval pass leaves multi-hop content unresolved.
How it connects
Connects iterative retrieval with RAG, context compression, reranking, hybrid search, multi-hop reasoning, Self-RAG, FLARE, and CRAG.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.