Routing selects the retrieval backend or knowledge source that should handle a query. In a multi-domain RAG system, an LLM classifies intent before retrieval so financial questions reach filing collections, product questions reach technical documentation, and out-of-domain or current questions reach web search. This avoids treating semantic similarity as proof that a source is authoritative, current, or appropriate. section 7.2
A practical routing design separates classification from retrieval and generation:
- Define explicit route labels around available sources, such as OpenAI documentation, 10-K filings, and live web search. section 7.2.2
- Ask an LLM to return a structured decision containing an action, a one-sentence reason, and optionally a direct short answer. section 7.2.2
- Map internal route labels to partitioned vector collections; embed the query with the same model used during ingestion before searching the selected collection. section 7.2.2
- Send web-search routes to live search, then synthesize a grounded response from retrieved results or collection chunks. section 7.2.2
- Record the selected route and reason so pipeline behavior can be audited, debugged, and improved. section 7.5
Three-way route decision
For “How do I create an OpenAI assistant with file search?”, select OPENAI_QUERY. For “What was Uber’s gross bookings in Q3 2021?”, select 10K_DOCUMENT_QUERY. For “What are the most popular open-source LLMs in 2025?”, select WEB_SEARCH because the answer requires current external information. The route is chosen before the corresponding collection search or web lookup. section 7.2.2
Routing belongs in a larger query lifecycle, but it is not always the first operation. A semantic-cache hit can return immediately, while a time-sensitive query can bypass the cache for live web search. Otherwise, routing selects the destination before query rewriting, decomposition, vector retrieval, and answer synthesis. Rewriting after route selection lets expansions and domain-specific wording fit the source being searched. section 7.5
Figure 7.6 A detailed, step-by-step pipeline diagram illustrating a query processing system
Trace how cache checks, time-sensitivity handling, routing, rewriting, decomposition, retrieval, synthesis, and caching fit into one query-processing pipeline. section 7.5

































