Routing decides who should handle a request before the work begins. A router classifies the input, then dispatches it to a suitable model, specialized agent, or workflow. Unlike a fixed pipeline that sends every request through the same steps, conditional routing chooses a branch based on the request. section 8.3.8 section 4.1.3
Use three questions to design a route:
- Do requests need different specialists? Technical and billing questions can go to different agents. If one generalist handles all inputs well, routing adds unnecessary overhead. section 4.1.3
- Is a cheaper model sufficient for this task? Sending simpler requests to cheaper models can reduce cost, but a wrong choice can lower the quality of the entire interaction. section 8.3.8
- How will you check the decision? Establish routing quality with a capable model before replacing it with a smaller router. Measure cost per session, resolved task, and user before optimizing. section 8.3.8
Routing inside retrieval-augmented generation
A retrieval-augmented generation (RAG) workflow can begin with triage, send a question to an appropriate retrieval branch, and pass retrieved passages to an answerer. Add that router only when branched retrieval is needed; if a one-shot RAG call works, skip the extra orchestration. The answerer should use the retrieved context, cite sources, and say when the available documents do not support an answer. section 11.3
The main failure mode is misclassification: even an excellent downstream agent may answer badly after receiving the wrong task. Requests spanning multiple domains are particularly difficult for a simple router. Prefer clear branches, observe answer accuracy, and keep the routing step only when its benefit justifies its overhead. section 4.1.3 section 11.3
