liveBook · Manning

LevelRAG

As explained in Retrieval Augmented Generation, The Foundational Ideas

LevelRAG is a retrieval approach that decomposes a complex question into atomic queries, routes them to searchers, summarizes results, and iterates until verification passes.

A single retrieval can miss part of a complex question even when the index is good. LevelRAG plans what to find before searching, then checks whether the gathered evidence covers the original question. It is suited to multi-part research and synthesis rather than a quick lookup. section 11.0 section 11.4

A High-Level Searcher decomposes the question into atomic information needs without searching an index itself. Low-Level Searchers retrieve evidence; the planner answers each atomic query from its documents, keeping the context focused instead of concatenating everything. A verification step checks the combined summaries. If coverage is incomplete, the planner issues supplemental queries and repeats, subject to a stopping limit. section 11.3.1 section 11.3.3

A compliance comparison

To compare 2025 UK and EU due-diligence requirements for domestic politically exposed persons, the planner asks separately for UK requirements and EU requirements. If UK evidence cites guidance FG25/3, it can add a query for that identifier—even though the user did not mention it. That follow-up distinguishes iterative planning from splitting a question just once. section 11.3.1

The retrieval method follows the information need:

  • Sparse search targets exact names and regulatory codes, with query operators to narrow or emphasize terms. section 11.3.2
  • Dense search targets broader concepts by embedding a generated hypothetical answer passage. section 11.3.2
  • Web search handles real-time needs or gaps in internal indices. section 11.3.2

The original LevelRAG design sends each atomic query through all Low-Level Searchers and summarizes their combined results. A cheaper implementation assigns each query one searcher in advance, but cannot compare which searcher returned the most useful evidence. Its engine dispatches queries, collects per-query summaries, and branches on a structured sufficiency verdict. section 11.3.3

Listing 11.5 The LevelRAG engine with verify-and-supplement loop.
class LevelRAGEngine:
    def __init__(self, hls: HighLevelSearcher,
                 low_level_searchers: dict, max_iter: int = 3):
        self.hls = hls
        self.searchers = low_level_searchers  #A
        self.max_iter = max_iter
 
    def execute(self, user_query: str) -> List[str]:
        summaries: List[str] = []
        pending = self.hls.decompose(user_query)
        for iteration in range(self.max_iter):
            for atomic in pending:  #B
                searcher = self.searchers.get(
                    atomic.tool, self.searchers['dense'])
                docs = searcher.retrieve(atomic.text)
                summary = self.hls.summarize(atomic, docs)
                summaries.append(
                    f"[{atomic.tool.upper()}] {atomic.rationale}\n{summary}"
                )
            verdict = self.hls.verify(user_query, summaries)  #C
            if verdict.sufficient:
                return summaries
            pending = self.hls.supplement(verdict.missing)  #D
            if not pending:
                break
        return summaries  #E
 

The engine listing shows dispatch, summary collection, verification, and a maximum-iteration guard—the control flow that makes this more than parallel multi-query retrieval. section 11.3.3

Choose this architecture when missed sub-questions are a greater risk than extra latency and integration work. For simpler fact checks, CRAG audits retrieval after it happens and can be cheaper on a successful first pass. The approaches can be combined by auditing each LevelRAG sub-result before summarization; production systems may reserve that full loop for complex queries. section 11.4 section 11.5.3

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Works through planner and searcher responsibilities, routing choices, summarization, and an iterative engine with a stopping limit.
Examples
Uses a UK–EU compliance comparison, a newly discovered guidance identifier, and Python planner and engine listings.
What it assumes
Assumes familiarity with RAG and retrieval indices while defining atomic queries and the planner–searcher split.
What you can do after
Build it from scratch as a routed, verify-and-supplement retrieval loop, or evaluate when its added cost is justified.
How it connects
Connects query decomposition, sparse retrieval, dense retrieval, web search, and CRAG-style corrective evaluation.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.