Routing sends a request to a model or agent suited to the work it requires. Sending every query to a powerful model raises cost and can increase latency as traffic grows; sending every query to a cheap model risks underserving difficult requests. A decision layer instead inspects each query before choosing where it goes. section 9.2.4
Two routing decisions serve different purposes:
- Model selection: send routine questions, such as store hours or shipping-label availability, to a smaller model; escalate complex or specialized questions to a more capable one. section 9.2.4
- Agent selection: direct a vague product request toward search, but an incomplete question about whether something fits toward policy answers. When intent is unclear, make a reasonable assumption and help the user narrow the request. section 8.2.2
A minimal model router
A simple Python decision function marks a query complex if it contains “legal,” “medical,” “regulatory,” “tax,” or “research,” or exceeds 30 words. The caller then selects gpt-5.4 for complex queries and a smaller model otherwise. These checks are routing heuristics, not proof that the selected model will answer correctly. section 9.2.4
def is_complex(query): #1
# Basic keyword check
advanced_keywords =
["legal", "medical", "regulatory", "tax", "research"] #2
if any(kw in query.lower() for kw in advanced_keywords): #3
return True #4
# If the query is very long, assume complexity
if len(query.split()) > 30: #5
return True #6
return False #7
def call_model(model_name, query): #8
# Stub for demonstration
return f"[{model_name}] response for: {query}" #9
def multi_model_router(query): #10
if is_complex(query): #11
return call_model("gpt-5.4", query) #12
else: #13
return call_model("gpt-5.4-mini", query) #14
# Example usage
print(multi_model_router("What time do you open?")) #15
print(multi_model_router("Could you summarize the new medical regulations regarding telehealth?")) #16
The listing separates complexity detection from the model call, making the selection rule easy to inspect and change. section 9.2.4
A classifier can also make the decision: one function-calling sketch asks a smaller model to return a simple-or-complex label, with an optional reason. Test routing as well as final responses. Ambiguous-intent tests check whether vague requests reach a useful agent and produce a helpful answer; evaluation and production monitoring remain important because efficient selection alone does not establish accuracy. section 9.2.4 section 8.2.2 section 9.0
