liveBook · Manning

Query decomposition

As explained in Retrieval Augmented Generation, The Foundational Ideas

Query decomposition is the technique of splitting a complex information need into smaller sub-queries that can be searched or answered separately.

A question that compares rules across jurisdictions may need facts from several documents. A single retrieval pass over the whole question can find one jurisdiction but miss the other, leaving the answer incomplete. Query decomposition splits the information need into smaller searches whose results can be combined. section 11.1.5

The basic approach creates sub-queries up front, retrieves for each, and fuses the results. An adaptive approach also checks what the results establish and searches again when evidence is missing. In LevelRAG, a High-Level Searcher plans the questions while lower-level searchers handle retrieval. section 11.1.5 section 11.3.1

Following a newly discovered lead

To compare enhanced due diligence for domestic PEPs in the UK and EU, search for each jurisdiction’s requirements separately. If a retrieved UK document cites FCA guidance FG25/3, turn that reference into another query. The new search follows evidence found during retrieval rather than merely splitting the original wording. section 11.3.1

A practical query-time loop is:

  • Identify the distinct facts needed to answer the question and issue focused retrieval tasks. section 11.3.1
  • Answer each sub-query from its retrieved documents, keeping the working context focused. section 11.3.1
  • Check whether those answers cover the original question; retrieve again where they do not. section 11.1.5 section 15.7.2
Figure 11.6 Solving the Compliance Scenario: How LevelRAG decomposes a complex query into atomic research tasks.

The compliance-scenario diagram shows a complex request becoming atomic research tasks. section 11.3.1

Decomposition can also be conditional rather than part of every request. In one production design, low initial retrieval scores trigger two or three targeted follow-up queries; a low-confidence answer can trigger another pass. This helped procedural questions whose answers were scattered across eligibility, application, and integration documents. Reserve the extra work for questions that need it: in an 85-question fintech evaluation, the full agentic pipeline improved Hit@5 from 54.12% to 62.35% while average latency rose from 0.79 to 5.02 seconds. Qualitative inspection identified sub-query generation as the largest contributor, not an ablation. section 15.7.2 section 15.7.4 section 15.9

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Details atomic queries, per-query summaries, coverage checks, and follow-up retrieval, but does not specify a query-generation algorithm.
Examples
Uses UK–EU compliance comparisons, CVaR and IRRBB searches, and a merchant-account procedure spread across documents.
What it assumes
Assumes familiarity with RAG retrieval and retrieved documents while illustrating atomic queries through a concrete comparison.
What you can do after
Apply it via LevelRAG’s High-Level Searcher or conditional sub-query generation, then check coverage and latency against a simpler retrieval pass.
How it connects
Connects multi-hop reasoning, RAG-Fusion, agentic RAG, retrieval evaluation, and re-ranking through the problem of incomplete evidence.

Also explained in

Compare how 3 books explain query decomposition →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.