Model selection chooses an LLM whose capabilities, cost, and deployment strategy fit the task. Production systems should avoid sending every query to the most capable model by default. section 13.4.1
A production selection strategy weighs:
- Task fit: match reasoning, coding, technical analysis, or domain specialization to the query. section 13.4.1
- Economics: compare token costs with infrastructure costs for open-weight deployments. section 13.4.1
- Operational complexity: routing can reduce costs by 60% to 80%, but adds endpoints, monitoring, versioning, tuning, and A/B testing. section 13.4.2
Three-tier query routing
A cost-optimized assistant uses a small, fast model to classify queries, a balanced model for routine generation, and a premium model for complex analysis. SIMPLE requests such as password resets use the cheapest tier; MODERATE requests use the balanced tier; COMPLEX requests receive premium reasoning. section 13.4.2
Listing 13.14 Cost-optimized DakkaBot with tiered model selection
class CostOptimizedDakkaBotCore:
def __init__(self):
self.router_llm = ChatGoogleGenerativeAI(
model="gemini-2.5-flash-lite", #1
temperature=0.0
)
self.generation_llm = ChatGoogleGenerativeAI(
model="gemini-2.5-flash", #2
temperature=0.0
)
self.premium_llm = ChatGoogleGenerativeAI(
model="gemini-2.5-pro", #3
temperature=0.0
)
async def route_query(self, query: str) -> str:
"""Use cheap model to classify query complexity"""
routing_prompt = f"""
Classify this query complexity: {query}
Options: SIMPLE, MODERATE, COMPLEX
SIMPLE: Basic factual questions, single-step procedures
MODERATE: Multi-step processes, troubleshooting
COMPLEX: Analysis, planning, integration across systems
Classification:""" #4
result = self.router_llm.invoke(routing_prompt) #5
return result.content.strip()
async def process_query_optimized(self, query: str) -> Dict[str, Any]:
complexity = await self.route_query(query) #6
if complexity == "SIMPLE":
llm = self.router_llm #7
elif complexity == "MODERATE":
llm = self.generation_llm
else:
llm = self.premium_llm
Inspect the tiered implementation, including separate router, generation, and premium model clients and minimal routing prompts. section 13.4.2
