Complexity-based routing selects a reasoning policy before committing resources. A request may go to deterministic code, a model and effort level, an evidence-gathering path, parallel search, or a human. Separate paths are useful when differences in requests produce measurable value in quality, cost, or latency. section 5.3
Difficulty chooses capacity; consequence chooses controls. A stronger model can help with hard analysis, but it cannot substitute for required evidence, authorization, approval, or containment. Make risk overrides explicit rather than hiding them in a single complexity label. section 5.3 section 5.3.5
A routing policy needs explicit operational ownership:
- Inspect represented facts, ambiguity, evidence needs, action risk, latency, and budget. Do not use request length alone as a difficulty proxy. section 5.3
- Own the thresholds, risk overrides, fallback, replay set, and upgrade policy. Repeated rescue by a deeper retry signals false simplicity. section 5.3
- Replay the router after material changes to models, prompts, tools, or traffic; its labels can drift, and classification itself consumes resources. section 5.3.5
Short does not mean safe
Sorting fifty records may be lengthy but deterministic. Deleting one production object may be short and clear but consequential. Likewise, a one-line payroll update can be easy to understand while still requiring verification and approval. section 5.3
Misroutes have asymmetric costs: a shallow path can silently mishandle hard analysis, while an unnecessarily capable model wastes resources. Skip routing when the workload is uniform or when classification overhead and misrouting erase the gain. Compare token costs only among designs that meet the same required outcome, authority boundary, and acceptance evidence. section 5.3 section 2.3.5
Table 5.5 Complexity-Based Routing trade-offs
|
Advantage
|
Limitation
|
|
Can reduce cost and latency while preserving measured quality
|
False-simple routes can fail silently
|
|
Makes model and policy choices explicit
|
Adds classification overhead to each routed request
|
|
Supports model upgrades through declarative configuration
|
Labels drift as models, tools, and traffic change
|
|
Can route to code, tools, models, or people
|
One difficulty label can hide consequence and evidence needs
|
Compare routing's potential savings and explicit policy choices with its overhead, drift, and false-simple failures. section 5.3.5
