liveBook · Manning

Routing

As explained in Retrieval Augmented Generation, The Foundational Ideas

A query router is a system component that analyzes a query and directs it to an appropriate retrieval or processing path.

A single retrieval path is a poor fit for every question. A factual lookup may need one search, while a comparison may need several searches and an ambiguous term may need clarification first. Routing adds a decision point that sends a query—or each part of it—to an appropriate processing path. section 15.2.2 section 15.2.4

The routing decision can be made at different levels:

  • Pipeline routing sends simple lookups through retrieve-and-generate, complex questions through planning and decomposition, and ambiguous questions to clarification before retrieval. section 15.2.2
  • Retriever routing selects a search method for each atomic sub-query. Dense search serves thematic similarity; sparse search serves exact entity matches. A LevelRAG implementation also offers web search. section 11.6 section 11.3.3
  • Dual-path routing can use both paths for one question. LightRAG extracts entity-oriented and theme-oriented keywords, searches graph and vector representations in parallel, then merges the results. section 12.4.2

A question with two retrieval needs

“Which suppliers were hit by the 2023 recall, and what does that say about our exposure?” calls for precise supplier and recall matches as well as broader evidence about supply-chain exposure. LightRAG sends those needs down its graph and vector paths rather than forcing the whole question into one bucket. section 12.4.2

A routing choice is not proof that the retrieved evidence answers the question. HybGRAG first chooses a graph module for a multi-hop question or a vector module for a simpler lookup; its critic can reject the result and reroute to the other module. LevelRAG instead decomposes a question, retrieves for its parts, and verifies whether the combined evidence is sufficient before stopping or searching for gaps. section 12.6.3 section 11.3.3

Listing 11.5 The LevelRAG engine with verify-and-supplement loop.
class LevelRAGEngine:
    def __init__(self, hls: HighLevelSearcher,
                 low_level_searchers: dict, max_iter: int = 3):
        self.hls = hls
        self.searchers = low_level_searchers  #A
        self.max_iter = max_iter
 
    def execute(self, user_query: str) -> List[str]:
        summaries: List[str] = []
        pending = self.hls.decompose(user_query)
        for iteration in range(self.max_iter):
            for atomic in pending:  #B
                searcher = self.searchers.get(
                    atomic.tool, self.searchers['dense'])
                docs = searcher.retrieve(atomic.text)
                summary = self.hls.summarize(atomic, docs)
                summaries.append(
                    f"[{atomic.tool.upper()}] {atomic.rationale}\n{summary}"
                )
            verdict = self.hls.verify(user_query, summaries)  #C
            if verdict.sufficient:
                return summaries
            pending = self.hls.supplement(verdict.missing)  #D
            if not pending:
                break
        return summaries  #E
 

The Python engine dispatches each atomic query by its assigned tool, collects summaries, and bounds the verification loop with a maximum iteration count. section 11.3.3

Choose the simplest route that meets the question’s needs. Routing to one searcher per sub-query is cheaper than querying every searcher, but commits to a modality before seeing which returns useful evidence. Feedback can catch some bad choices. It cannot, however, discover an unknown source: separately indexed enterprise systems still pose a problem when an answer spans a commit, a ticket, and a chat thread. section 11.3.3 section 15.8.4

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Explains pipeline, retriever, and dual-path decisions, then shows dispatch and bounded verification in a Python implementation.
Examples
Uses a supplier-recall question, complexity-based pipeline choices, and a Python LevelRAG engine.
What you can do after
Build it from scratch with per-sub-query searcher dispatch and a bounded verify-and-supplement loop.
How it connects
Connects query decomposition, dense retrieval, sparse retrieval, graph retrieval, and evidence verification.

Also explained in

Compare how 34 books explain routing →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.