A question that compares rules across jurisdictions may need facts from several documents. A single retrieval pass over the whole question can find one jurisdiction but miss the other, leaving the answer incomplete. Query decomposition splits the information need into smaller searches whose results can be combined. section 11.1.5
The basic approach creates sub-queries up front, retrieves for each, and fuses the results. An adaptive approach also checks what the results establish and searches again when evidence is missing. In LevelRAG, a High-Level Searcher plans the questions while lower-level searchers handle retrieval. section 11.1.5 section 11.3.1
Following a newly discovered lead
To compare enhanced due diligence for domestic PEPs in the UK and EU, search for each jurisdiction’s requirements separately. If a retrieved UK document cites FCA guidance FG25/3, turn that reference into another query. The new search follows evidence found during retrieval rather than merely splitting the original wording. section 11.3.1
A practical query-time loop is:
- Identify the distinct facts needed to answer the question and issue focused retrieval tasks. section 11.3.1
- Answer each sub-query from its retrieved documents, keeping the working context focused. section 11.3.1
- Check whether those answers cover the original question; retrieve again where they do not. section 11.1.5 section 15.7.2
Figure 11.6 Solving the Compliance Scenario: How LevelRAG decomposes a complex query into atomic research tasks.

The compliance-scenario diagram shows a complex request becoming atomic research tasks. section 11.3.1
Decomposition can also be conditional rather than part of every request. In one production design, low initial retrieval scores trigger two or three targeted follow-up queries; a low-confidence answer can trigger another pass. This helped procedural questions whose answers were scattered across eligibility, application, and integration documents. Reserve the extra work for questions that need it: in an 85-question fintech evaluation, the full agentic pipeline improved Hit@5 from 54.12% to 62.35% while average latency rose from 0.79 to 5.02 seconds. Qualitative inspection identified sub-query generation as the largest contributor, not an ablation. section 15.7.2 section 15.7.4 section 15.9


