Model selection turns production evidence into routing decisions. Quality, cost, latency, reliability, error rates, token usage, and cache behavior reveal which model or provider fits each workflow. section 3.8.3
Apply model selection through measurable operational criteria:
- Route classification tasks to a lower-cost model when it delivers comparable quality. section 3.8.3
- Adjust provider routing when rate limits or workflow-specific error rates make the primary choice unreliable. section 3.8.3
- Configure embedding model choice at the index level so every vector remains comparable. section 5.3.6
- Balance embedding dimensionality against retrieval quality, storage cost, and search latency. section 5.3.6
Evidence-based classification routing
If metrics show that Claude Sonnet matches GPT-4 quality for classification while costing 60% less, route classification requests to Claude and continue monitoring the result. section 3.8.3
Listing 5.12 Embedding generation through the Model Service
class EmbeddingGenerator:
def __init__(self, model_client): #A
self.model_client = model_client
def embed_chunks(
self,
chunks: List[Chunk],
model: str = "text-embedding-3-small",
batch_size: int = 100 #B
) -> List[List[float]]:
all_embeddings = []
for i in range(0, len(chunks), batch_size):
batch = chunks[i:i + batch_size]
texts = [chunk.text for chunk in batch]
response = self.model_client.embed( #C
texts=texts,
model=model
)
all_embeddings.extend(response.embeddings)
return all_embeddings
def embed_query(
self, query: str, model: str
) -> List[float]: #D
response = self.model_client.embed(
texts=[query],
model=model
)
return response.embeddings[0]Embedding generation exposes model choice, batching, and provider abstraction through a reusable Model Service interface. section 5.3.6











