liveBook · Manning

Abstraction

As explained in Designing AI Systems

Model abstraction is an interface design that hides provider-specific model details so applications can work with interchangeable models.

Model abstraction defines a consistent application-facing interface while hiding provider-specific model details. A Model Service can route requests across OpenAI, Anthropic, Google, self-hosted models, and future providers without requiring application rewrites. section 1.3 section 3.2

A production-oriented abstraction should provide:

  • Adapters that translate the shared contract into each provider's request and response format. section 3.2 section 3.2.1
  • Intelligent routing based on task requirements, automatic fallback during provider failures, and cost-aware model selection. section 1.3
  • Configuration-driven provider selection so teams can evaluate models, run A/B tests, and change vendors without changing application logic. section 3.2.1
Table 3.1 Variations in LLM provider formats
Aspect
OpenAI
Anthropic
Google (Gemini)
System prompt
In messages array as role: "system"
Separate system parameter
In messages array
Message structure
{role, content}
{role, content}
{role, parts}
Response location
choices[0].message.content
content blocks array
candidates[0].content
Token field names
prompt_tokens, completion_tokens
input_tokens, output_tokens
promptTokenCount, candidatesTokenCount

Compare how providers vary in system-prompt placement, message structure, response locations, and token field names—the differences the abstraction must conceal. section 3.2.1

How Designing AI Systems covers it

How deep it goes
Explains the shared interface, adapters, provider-format differences, routing, and fallback, but does not provide adapter code.
Examples
Contrasts OpenAI, Anthropic, and Google Gemini request, response, system-prompt, and token formats.
What it assumes
Assumes familiarity with AI providers, application logic, APIs, and configuration-based service integration.
What you can do after
Apply it via a Model Service to add providers, evaluate models, and switch configurations without rewriting application logic.
How it connects
Connects model abstraction to vendor lock-in, adapters, configuration, intelligent routing, fallback, A/B testing, and cost optimization.

Also explained in

Compare how 24 books explain abstraction →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.