A prompt asking for code may need a different response path from one asking for poetry or a simple explanation. Semantic routing, also called intent routing, predicts which language model is likely to work best for an input, sends the request there, and processes the response. It moves that choice out of a single model’s prompt instructions and into routing logic. section 10.1.2 section 10.1.3
A practical router follows three steps:
- Collect prompts that have worked well for users as training data. section 10.1.3
- Train a classifier to predict which LLM is likely to produce the best result for a new prompt. Semantic search, or vector search, is a core technology behind this approach. section 10.1.3
- Use the prediction in routing logic—such as Python if–elif branches—to request a response from the selected model and process its output. section 10.1.2 section 10.1.3
Different requests, different paths
Consider requests to generate code, write poetry, and explain AI to a five-year-old. Rather than send all three to one LLM with increasingly elaborate instructions, a router classifies each request and selects the LLM predicted to handle it best. The prediction depends on the router’s training data; the prompt alone does not establish which model will win. section 10.1.3
Routing can improve the cost–benefit balance of an LLM service, but selection is not a guarantee of a good or safe answer. Models have different guardrails and quirks, so the chosen path still needs appropriate handling. section 10.1.3
