Chapter 9: Router: Classifying and dispatching
The introduction explains why a single general-purpose agent can struggle with specialized tasks and presents Router as a unified interface that classifies requests and dispatches them to suitable specialists.
After this chapter you will be able to
- General agent limitations. Explain why giving one agent every support task, tool, and policy can lead to poor tool choices, mixed-up domain rules, and an inability to optimize each task independently.
- Router classification and dispatch. Describe how the Router presents one external interface while a selector classifies an incoming task, forwards it to an appropriate sub-agent, and returns that specialist's response. Explain that the specialist, not the router, does the task using its own tools, knowledge, and policies.
- Limits of fixed hierarchies. Explain why a fixed hierarchy can be a poor fit for an agent with many specialized components: they may be needed in no predetermined order and may work on diverse, interdependent payloads.
- Router as grouped interface. Describe the Router pattern's proposed solution: put a routing mechanism around a group of specialized components so the rest of the agent sees the group as a single, general agent.
- Component selection and payload. Explain that a router chooses one component to handle an incoming task and constructs the payload that component expects. Describe how it may try another component if the first cannot complete the task.
- Forwarding and common interface. Describe how a router forwards all or most input information in the chosen component's expected form and passes its output to the user without alteration. Distinguish this from incrementally solving a task through component subtasks.
- Router timing and benefits. Explain that routers are usually introduced after specialized components have increased an agent’s Capability, and that routers primarily improve Robustness and Reliability while often making the design and code clearer.
- Key based routing flow. Trace how the selector IE classifies a task and returns a key, then how the Router uses that key to find a sub-agent in its registry and forward the payload.
- Specialized domain triage. Explain how a router interprets a user's request and sends it to a domain-specific sub-agent, keeping a modular system from becoming one general-purpose agent with too many tools.
- Complexity-based inference budgeting. Describe how a router can assess request complexity with a lightweight classifier, send simple tasks to inexpensive, low-latency models, and reserve more capable models for deeper analysis.
- Security privilege routing. Explain how a router can detect requests needing special privileges, direct users through verification, and keep high-privilege agents' data and conversational memories separate from general-purpose processing.
- Router versus Guardrail decisions. Distinguish Router’s choice among several options from Guardrail’s choice of whether to invoke a component.
- Router versus Orchestrator responsibilities. Distinguish Router’s dispatch role from Orchestrator’s task subdivision and maintenance of a work context containing responses from tools or sub-agents.
- Proximity routers versus agent swarms. Explain that proximity-router sub-agents collaborate only on routing, after which the chosen sub-agent executes the entire task; in an agent swarm, multiple agents collaborate on the task itself.
- Top-k routing and reconciliation. Trace how a selector IE ranks sub-agent identifiers, dispatches a task to the top k candidates in parallel, and uses a reconciler to produce one final output through synthesis or majority-based reduction.
- Hierarchical task dispatch. Trace how a root router selects a child router, which makes a narrower selection until a terminal specialist agent handles the task.
- Shared interface and compositionality. Explain how treating nested routers and specialist agents as IE objects lets a parent router handle either kind of child through the same interface, enabling recursive traversal of the routing tree.
- Neighbor-based task routing. Explain how a proximity router decentralizes routing: an initially selected sub-agent can decide it cannot handle a task and pass it to a nearby specialist, even after starting work.
- Routing loop prevention. Describe how local routing decisions can send a task around a cycle, and how carrying a traversal list or TTL counter with the task can limit repeated hops.
- Cross-jurisdiction tax rule risk. Explain why a general-purpose tax advisor can be unreliable when it draws on rules from conflicting jurisdictions, such as applying Swiss VAT rules to a US sales tax question.
- Jurisdictional triage and specialist dispatch. Trace how the selector IE identifies both the country and legal category, then forwards the tax question to a matching sub-agent backed by fine-tuning or RAG access to that country's tax code. Explain how this separation provides specialist coverage without one massive, potentially conflicting context window.
- Complexity-based model dispatch. Describe how a lightweight, low-cost inference engine grades incoming tasks by reasoning complexity and routes simple tasks to small or local models while escalating complex tasks to high-parameter frontier models.
- Parallel specialist selection. Describe how an ensemble router selects the k most relevant specialists, invokes them in parallel, and has a final recomposer synthesize their findings.
What it builds on
- How the Guardrail pattern works (chapter 4) · How Guardrail gates a component.
- Runtime and orchestration (chapter 2) · Runtime and orchestration of agent components.
- How the Voter pattern works (chapter 6) · the Voter pattern's use of multiple outputs
Section by section
Introduction
The introduction explains why a single general-purpose agent can struggle with specialized tasks and presents Router as a unified interface that classifies requests and dispatches them to suitable specialists.
- General agent limitations: Explain why giving one agent every support task, tool, and policy can lead to poor tool choices, mixed-up domain rules, and an inability to optimize each task independently.
Why it matters: These failure modes explain why the system needs routing rather than simply a larger general-purpose agent. - Router classification and dispatch: Describe how the Router presents one external interface while a selector classifies an incoming task, forwards it to an appropriate sub-agent, and returns that specialist's response. Explain that the specialist, not the router, does the task using its own tools, knowledge, and policies.
Why it matters: Separating dispatch from execution is the central architectural move of the Router pattern. - Routing oversight chokepoint: Explain how sending every request through the selector creates a point for logging decisions, flagging ambiguous cases, and enforcing access controls before a specialist receives the task.
Why it matters: Routing can provide a place to monitor and control access, not just improve task matching.
Key terms: tool choice overload, knowledge cross-contamination, specialization, Router, selector, sub-agent, classification, routing, governance, observability
9.1 The pain point: Too many specialized components to choose from
This section explains why specialized agent components are difficult to organize in a fixed hierarchy and introduces the Router pattern as a way to present them as one general agent.
- Limits of fixed hierarchies: Explain why a fixed hierarchy can be a poor fit for an agent with many specialized components: they may be needed in no predetermined order and may work on diverse, interdependent payloads.
Why it matters: The routing pattern addresses an organization problem created by specialization. - Router as grouped interface: Describe the Router pattern's proposed solution: put a routing mechanism around a group of specialized components so the rest of the agent sees the group as a single, general agent.
Why it matters: This is the section's central idea for accommodating multiple specialized components.
Key terms: specialized components, fixed hierarchy, interdependent payloads, Router pattern, routing mechanism, single, general agent
9.2 How the Router pattern works
This section explains how a Router chooses a specialized component, prepares its input, and presents the result through a general interface.
- Component selection and payload: Explain that a router chooses one component to handle an incoming task and constructs the payload that component expects. Describe how it may try another component if the first cannot complete the task.
Why it matters: Routing involves preparing a usable request, not merely naming a destination. - Forwarding and common interface: Describe how a router forwards all or most input information in the chosen component's expected form and passes its output to the user without alteration. Distinguish this from incrementally solving a task through component subtasks.
Why it matters: These boundaries distinguish a router from an agent that assembles an answer across multiple component calls. - Choice criteria and selectors: Explain that component choice can reflect task nature, complexity, or security demands, matching expertise, efficiency, or information sensitivity. State that selection may use simple heuristics but is more often performed by a specialized IE that also constructs the payload.
Why it matters: The choice criteria explain why different tasks should go to different components.
Key terms: router, payload, fallbacks, selector element, input forwarding, output forwarding, interface generality, constraint preference choice, task-component expertise alignment, task-component efficiency alignment
9.2.1 Tradeoffs
This section explains when routers are typically introduced and the benefits they bring to an agent.
- Router timing and benefits: Explain that routers are usually introduced after specialized components have increased an agent’s Capability, and that routers primarily improve Robustness and Reliability while often making the design and code clearer.
Why it matters: This distinguishes the capability gained from specialized components from the benefits of adding a router.
Key terms: routers, specialized components, Capability, Robustness, Reliability, design and code clarity
9.2.2 Implementation sketch
The section sketches a Router that classifies a task, uses the resulting key to dispatch its payload to a sub-agent, and handles an unrecognized selection.
- Key based routing flow: Trace how the selector IE classifies a task and returns a key, then how the Router uses that key to find a sub-agent in its registry and forward the payload.
Why it matters: This two-stage flow is the essential behavior of the Router implementation. - Decoupled execution and selection misses: Explain that the Router need not know its sub-agents' internal logic, and describe what this simple implementation does when a selected key is missing versus the alternatives mentioned.
Why it matters: Separating selection from execution makes the dispatch role clear, while missing keys require an explicit response.
Key terms: selector IE, key-based mapping, sub_agents, registry, payload, UnknownSubAgentError, selection miss, general-purpose catch-all agent, retrial
9.2.3 When and where to use Router
The section explains when routers help agentic systems by directing requests to specialized agents, appropriately priced models, available instances, or secure workflows.
- Specialized domain triage: Explain how a router interprets a user's request and sends it to a domain-specific sub-agent, keeping a modular system from becoming one general-purpose agent with too many tools.
Why it matters: Sending each request to the right specialist limits irrelevant tool choices and reduces the risk of mixing up domain-specific rules. - Complexity-based inference budgeting: Describe how a router can assess request complexity with a lightweight classifier, send simple tasks to inexpensive, low-latency models, and reserve more capable models for deeper analysis.
Why it matters: Matching model cost to task difficulty preserves the system's reasoning budget. - Health-aware load balancing: Describe how a router distributes requests among instances of an agent using availability signals such as endpoint saturation and health status.
Why it matters: Routing toward available instances helps maintain response speed during surges and avoids overloading one instance. - Security privilege routing: Explain how a router can detect requests needing special privileges, direct users through verification, and keep high-privilege agents' data and conversational memories separate from general-purpose processing.
Why it matters: A router can serve as a governance boundary before sensitive requests reach restricted agents or data.
Key terms: router, specialized domain triage, sub-agents, tool catalog bloat, inference budget optimization, lightweight semantic classifier, reasoning budget, low-latency models, frontier models, load balancing
9.2.4 How Router works with other patterns
This section distinguishes Router from Guardrail, Orchestrator, and agent swarms by what each component chooses or manages.
- Router versus Guardrail decisions: Distinguish Router’s choice among several options from Guardrail’s choice of whether to invoke a component.
Why it matters: The distinction clarifies when a system needs dispatch rather than a gate. - Router versus Orchestrator responsibilities: Distinguish Router’s dispatch role from Orchestrator’s task subdivision and maintenance of a work context containing responses from tools or sub-agents.
Why it matters: Routing a task is narrower than managing its execution across smaller pieces. - Proximity routers versus agent swarms: Explain that proximity-router sub-agents collaborate only on routing, after which the chosen sub-agent executes the entire task; in an agent swarm, multiple agents collaborate on the task itself.
Why it matters: The distinction identifies whether collaboration determines an assignee or performs the work.
Key terms: Router, Guardrail, Orchestrator, work context, tools/sub-agents, proximity routers, agent swarms, flat teams, sub-agent
9.3.1 Ensemble router
This section shows how an ensemble router selects several specialists for a task, runs them in parallel, and reconciles their outputs.
- Top-k routing and reconciliation: Trace how a selector IE ranks sub-agent identifiers, dispatches a task to the top k candidates in parallel, and uses a reconciler to produce one final output through synthesis or majority-based reduction.
Why it matters: The defining difference from ordinary routing is that several selected specialists contribute to the answer. - Dynamic specialist committee: Explain why selecting multiple specialists can help when the best route is ambiguous or a task warrants hedging, and distinguish this dynamic Voter from a fixed committee.
Why it matters: It clarifies when the extra parallel work is worth considering and why the ensemble changes from task to task.
Key terms: ensemble router, selector IE, top-k, sub-agent identifiers, reconciler element, dynamic Voter, committee, hedging, optimal path
9.3.2 Multi-layered router
This section shows how routers can form a hierarchy that narrows a task from broad categories to a terminal specialist agent.
- Hierarchical task dispatch: Trace how a root router selects a child router, which makes a narrower selection until a terminal specialist agent handles the task.
Why it matters: This is the dispatch path that distinguishes a multi-layered router from a router that selects a specialist outright. - Shared interface and compositionality: Explain how treating nested routers and specialist agents as IE objects lets a parent router handle either kind of child through the same interface, enabling recursive traversal of the routing tree.
Why it matters: A shared interface makes it possible to add routing layers without changing how a parent delegates. - Local selection benefits: Justify why organizing specialists under successive routers makes selection more transparent and reduces selector-prompt complexity: each router needs to distinguish only among its immediate children, not every agent in the system.
Why it matters: Local choices make a large collection of specialists easier to organize and route among.
Key terms: multi-layered router, root router, sub-router, terminal sub-agent, coarse-to-fine routing, IE objects, shared interface, Composite OOP pattern, compositionality, selector prompts
9.3.3 Proximity router
This section explains how a proximity router passes tasks among neighboring specialists and how to prevent tasks from cycling indefinitely.
- Neighbor-based task routing: Explain how a proximity router decentralizes routing: an initially selected sub-agent can decide it cannot handle a task and pass it to a nearby specialist, even after starting work.
Why it matters: A task need not fail just because its first destination was not the best specialist. - Routing loop prevention: Describe how local routing decisions can send a task around a cycle, and how carrying a traversal list or TTL counter with the task can limit repeated hops.
Why it matters: Without a way to stop cycles, a distributed routing network can overflow the call stack instead of completing the task.
Key terms: proximity router, sub-agent, neighbors, RoutingEncapsulation, local selector IE, chain of responsibility, cyclical references, local knowledge, RecursionError, stack overflow
9.4.1 Specialized jurisdiction-aware tax advisor
This section uses a jurisdiction-aware tax advisor to show how a Router can prevent tax rules from different legal domains from being mixed.
- Cross-jurisdiction tax rule risk: Explain why a general-purpose tax advisor can be unreliable when it draws on rules from conflicting jurisdictions, such as applying Swiss VAT rules to a US sales tax question.
Why it matters: Using the wrong jurisdiction's rules can cause serious legal and financial errors. - Jurisdictional triage and specialist dispatch: Trace how the selector IE identifies both the country and legal category, then forwards the tax question to a matching sub-agent backed by fine-tuning or RAG access to that country's tax code. Explain how this separation provides specialist coverage without one massive, potentially conflicting context window.
Why it matters: Routing each question to an appropriate specialist reduces rule mixing without requiring one model to carry every tax code in its context.
Key terms: cross-contamination of knowledge, jurisdiction, Swiss VAT, United States sales tax, selector IE, jurisdictional triage, legal category, sub-agent, fine-tuned, RAG-access
9.4.2 Complexity-driven inference budget optimizer
This section shows how a Router can reduce inference costs by sending simple tasks to inexpensive models and complex tasks to frontier models.
- Complexity-based model dispatch: Describe how a lightweight, low-cost inference engine grades incoming tasks by reasoning complexity and routes simple tasks to small or local models while escalating complex tasks to high-parameter frontier models.
Why it matters: The routing decision is what lets an agent reserve expensive inference for tasks that need it. - Inference budget rationale: Explain why routing by complexity is useful in large-scale deployments: using a frontier model for every interaction is prohibitively expensive and often unnecessary.
Why it matters: The cost rationale explains why adding a complexity filter is worth considering.
Key terms: Router, IE, selector, reasoning complexity, frontier models, inference budget, frontier model, complexity filter
9.4.3 Ensemble-based medical symptom triage
This section shows how an ensemble router handles ambiguous medical symptoms by consulting multiple specialists and combining their findings.
- Parallel specialist selection: Describe how an ensemble router selects the k most relevant specialists, invokes them in parallel, and has a final recomposer synthesize their findings.
Why it matters: Routing to more than one specialist lets the system consider distinct interpretations of the same symptoms. - Ambiguous symptom coverage: Explain why a single routing choice can be too reductive when symptoms are ambiguous and choosing the wrong path has significant consequences, using chest tightness and the cardiology and anxiety possibilities as the example.
Why it matters: Considering both paths can preserve important possibilities that a single-choice router would miss.
Key terms: Ensemble Router, k most relevant specialists, selector IE, final recomposer, persistent chest tightness, cardiology specialist, anxiety/mental health specialist, traditional linear router
9.5 Summary
This summary presents Router as a disciplined way to dispatch tasks among specialized components through one general interface, distinguishes it from an Orchestrator, and recaps its variants and examples.
Concepts explained in this chapter
- subagentAn agent that operates under the coordination or supervision of another agent.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
