liveBook · Manning

Routing

As explained in Build an Advanced RAG Application (From Scratch)

A query router is a system component that determines which retrieval backend or knowledge source should handle a user query.

Routing selects the retrieval backend or knowledge source that should handle a query. In a multi-domain RAG system, an LLM classifies intent before retrieval so financial questions reach filing collections, product questions reach technical documentation, and out-of-domain or current questions reach web search. This avoids treating semantic similarity as proof that a source is authoritative, current, or appropriate. section 7.2

A practical routing design separates classification from retrieval and generation:

  • Define explicit route labels around available sources, such as OpenAI documentation, 10-K filings, and live web search. section 7.2.2
  • Ask an LLM to return a structured decision containing an action, a one-sentence reason, and optionally a direct short answer. section 7.2.2
  • Map internal route labels to partitioned vector collections; embed the query with the same model used during ingestion before searching the selected collection. section 7.2.2
  • Send web-search routes to live search, then synthesize a grounded response from retrieved results or collection chunks. section 7.2.2
  • Record the selected route and reason so pipeline behavior can be audited, debugged, and improved. section 7.5

Three-way route decision

For “How do I create an OpenAI assistant with file search?”, select OPENAI_QUERY. For “What was Uber’s gross bookings in Q3 2021?”, select 10K_DOCUMENT_QUERY. For “What are the most popular open-source LLMs in 2025?”, select WEB_SEARCH because the answer requires current external information. The route is chosen before the corresponding collection search or web lookup. section 7.2.2

Routing belongs in a larger query lifecycle, but it is not always the first operation. A semantic-cache hit can return immediately, while a time-sensitive query can bypass the cache for live web search. Otherwise, routing selects the destination before query rewriting, decomposition, vector retrieval, and answer synthesis. Rewriting after route selection lets expansions and domain-specific wording fit the source being searched. section 7.5

Figure 7.6 A detailed, step-by-step pipeline diagram illustrating a query processing system

Trace how cache checks, time-sensitivity handling, routing, rewriting, decomposition, retrieval, synthesis, and caching fit into one query-processing pipeline. section 7.5

How Build an Advanced RAG Application (From Scratch) covers it

How deep it goes
Builds a three-route LLM classifier through prompt schema, collection dispatch, retrieval, and orchestration.
Examples
Demonstrates OpenAI file search, Uber financials, current LLM popularity, and other representative route decisions.
What it assumes
Assumes indexed document collections and introduces routing incrementally alongside embeddings, retrieval, and response generation.
What you can do after
Build it from scratch as an LLM-routed RAG dispatcher for partitioned Qdrant collections and web search.
How it connects
Connects semantic caching, time sensitivity, query rewriting, query decomposition, vector search, and grounded response generation.

Also explained in

Compare how 34 books explain routing →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build an Advanced RAG Application (From Scratch). The section links point to the parts of the book it draws on.