liveBook · Manning

Two-pass routing

As explained in Building Reliable AI Systems

A method that evaluates a task with one model, then escalates to another model if the first result is inadequate.

Two-pass routing tries a query with a cheaper model before deciding whether a stronger model is needed. If the first answer appears incomplete, incoherent, or low-confidence, the system escalates the query. This keeps routine requests on the cheaper path while giving difficult requests another chance. section 9.2.4 section 9.2.6

The decision happens after the first answer:

  • Send the query to the cheaper model and inspect its response for adequacy. section 9.2.4
  • Keep an adequate answer; otherwise, send the query to the stronger model. section 9.2.6

A borderline query

Try gpt-5.4-mini first. If its response has low confidence or coherence, escalate to gpt-5.4. A query whose first answer is adequate avoids the stronger-model call. section 9.2.6

This differs from routing solely on the incoming query: keyword rules, a classifier, or a smaller model using function calling can choose a destination before an answer exists. Two-pass routing instead uses the first answer as evidence. No specific confidence metric or escalation threshold is prescribed, so that decision needs its own evaluation. section 9.2.4 section 9.2.6

How Building Reliable AI Systems covers it

How deep it goes
Defines the fallback sequence and qualitative triggers but specifies no confidence metric or escalation threshold.
Examples
Uses a borderline query tried with gpt-5.4-mini before possible escalation to gpt-5.4.
What you can do after
Operate/configure a two-model fallback that reserves stronger-model calls for first answers judged inadequate.
How it connects
Connects multimodel fallback with confidence estimation and contrasts response-based escalation with query-complexity routing.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Building Reliable AI Systems. The section links point to the parts of the book it draws on.