liveBook · Manning

Prompt caching

Also called prefix caching

Prompt caching stores and reuses previously processed prompt content or computations to reduce repeated processing, cost, or latency.

How 3 Manning books explain it

Designing AI Systems

Prefix caching is a caching method that reuses computations for an identical leading portion of a prompt.

How it is used
Reduces model-serving cost for repeated prompt prefixes after application-level response caching misses.
How deep it goes
Explains caching layers, provider adapters, cache controls, keying, eviction, and monitoring, but not provider cache internals.
Examples
Uses FAQ systems, stable system prompts, repeated document context, Anthropic markers, OpenAI automatic caching, and vLLM prefix caching.
What you can do after
Operate/configure provider caching through adapters, request options, cache policies, and cost-effectiveness metrics.

Read how Designing AI Systems explains prompt caching →

LLM Customization and Fine-Tuning

A technique that stores reusable prompt prefixes or inputs so repeated content does not need to be processed or billed in the same way each time.

How it is used
Uses repeated-prefix discounts to refine managed-API costs before comparing them with self-hosting or adaptation.
How deep it goes
Quantifies provider discounts and explains which prompt content benefits, without detailing cache implementation.
Examples
Uses a 3,000-token demonstration block and contrasts reusable prefixes with changing retrieved chunks.
What you can do after
Evaluate whether repeated prompt prefixes can materially reduce managed-API costs before comparing self-hosting.

Read how LLM Customization and Fine-Tuning explains prompt caching →

Context Engineering

Prompt caching reuses previously computed representations of prompt context, particularly shared prefixes, to avoid processing them again.

How it is used
Enables reuse of shared knowledge prefixes in cache-augmented generation to reduce request processing cost and latency.
How deep it goes
Explains prefix reuse, cache expiry and miss handling, alongside serving-engine memory optimizations, without provider-specific implementation details.
Examples
Illustrates warming an external-knowledge prefix and appending subsequent user queries for fresh completions.
What you can do after
Evaluate cache reuse and plan for expiry and misses when managing shared knowledge prefixes.

Read how Context Engineering explains prompt caching →

Also covered in

What to know first

Related concepts

Questions about prompt caching

What is prompt caching?

Prompt caching stores and reuses previously processed prompt content or computations to reduce repeated processing, cost, or latency.

Which Manning books explain prompt caching?

Designing AI Systems, LLM Customization and Fine-Tuning and Context Engineering.

What should I know before learning prompt caching?

Manning books that teach prompt caching usually introduce token, cost, expiry and language model first.

What is prompt caching a kind of?

Prompt caching is a kind of caching. Caching is the temporary storage of data or computed results for reuse, reducing the work or delay of retrieving or computing them again.

What is prompt caching also called?

Prompt caching is also called prefix caching.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.