A managed API is an externally hosted model accessed through an HTTP endpoint. Also called model-as-a-service or hosted inference, it lets a team send prompts and pay per token while the provider handles infrastructure, updates, reliability, and scaling. section 2.4.1
The decision is whether faster delivery and lower operational burden outweigh vendor dependence and the need to send data to the provider. Vendor model versions, deprecations, pricing, and terms remain management concerns even when infrastructure maintenance is outsourced. section 2.4.1 section 2.4
Start with a managed API when:
- The problem is still being explored: prompt iteration can reveal capability gaps quickly without a training run. section 2.4.1
- A hosted model can solve the task without fine-tuning, as with summarization, translation, code generation, or questions over provided context. section 2.4.1
- The team wants to avoid managing GPU clusters, CUDA configuration, deployments, and latency monitoring. section 2.4.1
Listing 2.1 Calling a managed API (Anthropic Python SDK)
import anthropic
client = anthropic.Anthropic()
message = client.messages.create(
model="claude-opus-5-5",
max_tokens=1000,
messages=[
{
"role": "user",
"content": "Explain the difference between a stock and a bond "
"in two sentences.",
}
],
)
print(message.content[0].text)This Anthropic Python SDK call sends a prompt to a hosted model. It requires the SDK and an ANTHROPIC_API_KEY environment variable; other providers use their own clients, credentials, and model identifiers. section 2.4.1
Reconsider self-hosting when hard privacy or residency requirements prohibit sending data to a vendor, deeper customization or weight access is necessary, or traffic makes owned infrastructure cheaper beyond a break-even point. Self-hosting also brings an ongoing serving and maintenance commitment. section 2.4.1 section 2.11 section 2.4
Table 2.3 Buy versus build: criteria for choosing between a managed API and a self-hosted model.
|
Criterion
|
Buy: managed API
|
Build: self-hosted
|
|
Setup time
|
Minutes
|
Days to weeks to provision GPUs and
configure the serving stack
|
|
Upfront cost
|
Low, pay per token
|
High, physical GPU hardware or cloud
GPU VMs
|
|
Cost at scale
|
Grows linearly with usage
|
One-time infrastructure cost amortized
across requests
|
|
Customization
|
Prompt engineering plus limited
fine-tuning APIs
|
Fine-tuning, weight access, custom
serving stack
|
|
Data privacy
|
Sent to vendor servers
|
Stays on your infrastructure
|
|
Model choice
|
Vendor decides versions and
deprecations
|
Any open-weight model you choose
|
|
Latency control
|
Vendor SLA on shared infrastructure
|
You own the hardware and networking
path
|
|
Maintenance burden
|
Zero infrastructure burden; vendor
management overhead (deprecations, pricing, terms) is real.
|
Ongoing operational commitment for
hardware, serving stack, and scaling
|
Compare setup time, cost, customization, privacy, latency control, and maintenance across managed and self-hosted models. section 2.4

