Prompt caching reduces the cost of repeated prompt prefixes—such as system prompts and few-shot examples—when using a managed API. Savings vary by provider and matter most when repeated input is a substantial share of the prompt. section 2.4.1
Check what repeats before estimating the savings:
- A repeated 3,000-token demonstration block can roughly halve the bill; short prompts save less when output tokens dominate. section 2.4.1
- Retrieved chunks change with each query, so prompt caching does not discount them. section 2.4.1
Figure 2.2 Managed-API cost scales with request volume; self-hosted cost is largely fixed.

Use the cost comparison to weigh managed-API discounts, including caching, against largely fixed self-hosting costs. section 2.4.1


