liveBook · Manning

Complexity-based routing

As explained in Designing AI Agents

A routing pattern that selects an agent's reasoning policy and resource path according to request properties such as difficulty and consequence.

Complexity-based routing selects a reasoning policy before committing resources. A request may go to deterministic code, a model and effort level, an evidence-gathering path, parallel search, or a human. Separate paths are useful when differences in requests produce measurable value in quality, cost, or latency. section 5.3

Difficulty chooses capacity; consequence chooses controls. A stronger model can help with hard analysis, but it cannot substitute for required evidence, authorization, approval, or containment. Make risk overrides explicit rather than hiding them in a single complexity label. section 5.3 section 5.3.5

A routing policy needs explicit operational ownership:

  • Inspect represented facts, ambiguity, evidence needs, action risk, latency, and budget. Do not use request length alone as a difficulty proxy. section 5.3
  • Own the thresholds, risk overrides, fallback, replay set, and upgrade policy. Repeated rescue by a deeper retry signals false simplicity. section 5.3
  • Replay the router after material changes to models, prompts, tools, or traffic; its labels can drift, and classification itself consumes resources. section 5.3.5

Short does not mean safe

Sorting fifty records may be lengthy but deterministic. Deleting one production object may be short and clear but consequential. Likewise, a one-line payroll update can be easy to understand while still requiring verification and approval. section 5.3

Misroutes have asymmetric costs: a shallow path can silently mishandle hard analysis, while an unnecessarily capable model wastes resources. Skip routing when the workload is uniform or when classification overhead and misrouting erase the gain. Compare token costs only among designs that meet the same required outcome, authority boundary, and acceptance evidence. section 5.3 section 2.3.5

Table 5.5 Complexity-Based Routing trade-offs
Advantage
Limitation
Can reduce cost and latency while preserving measured quality
False-simple routes can fail silently
 
Makes model and policy choices explicit
 
Adds classification overhead to each routed request
Supports model upgrades through declarative configuration
 
Labels drift as models, tools, and traffic change
 
Can route to code, tools, models, or people
One difficulty label can hide consequence and evidence needs

Compare routing's potential savings and explicit policy choices with its overhead, drift, and false-simple failures. section 5.3.5

How Designing AI Agents covers it

How deep it goes
Details routing features, thresholds, risk overrides, fallback, failure modes, and replay after upgrades.
Examples
Contrasts record sorting, production deletion, and a payroll update; also selects routing for a code-review agent.
What you can do after
Configure routing features, thresholds, overrides, fallbacks, and replay policies while accounting for misrouting and overhead.
How it connects
Connects routing to governance, Context Triage, Generator-Critic, and Parallel Exploration.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Agents. The section links point to the parts of the book it draws on.