Model abstraction defines a consistent application-facing interface while hiding provider-specific model details. A Model Service can route requests across OpenAI, Anthropic, Google, self-hosted models, and future providers without requiring application rewrites. section 1.3 section 3.2
A production-oriented abstraction should provide:
- Adapters that translate the shared contract into each provider's request and response format. section 3.2 section 3.2.1
- Intelligent routing based on task requirements, automatic fallback during provider failures, and cost-aware model selection. section 1.3
- Configuration-driven provider selection so teams can evaluate models, run A/B tests, and change vendors without changing application logic. section 3.2.1
Table 3.1 Variations in LLM provider formats
|
Aspect
|
OpenAI
|
Anthropic
|
Google
(Gemini)
|
|
System
prompt
|
In messages
array as role: "system"
|
Separate system
parameter
|
In messages
array
|
|
Message
structure
|
{role, content}
|
{role, content}
|
{role, parts}
|
|
Response
location
|
choices[0].message.content
|
content
blocks array
|
candidates[0].content
|
|
Token field
names
|
prompt_tokens, completion_tokens
|
input_tokens, output_tokens
|
promptTokenCount, candidatesTokenCount
|
Compare how providers vary in system-prompt placement, message structure, response locations, and token field names—the differences the abstraction must conceal. section 3.2.1























