Cache-augmented generation
A generation technique that reuses cached information, context, or computation to reduce repeated retrieval or inference costs.
Retrieval Augmented Generation, The Foundational Ideas
A generation approach that augments retrieval with cached information or computation to improve efficiency and reduce inference cost.
- How it is used
- Optimizes production retrieval-augmented generation by improving retrieval efficiency and reducing inference cost.
- How deep it goes
- Positions caching as an architectural implementation detail and selection aid, without specifying cache algorithms or operational procedures.
- Examples
- Uses small fixed corpora and compares long-context with retrieval performance across 2,326 cases and eleven models.
- What you can do after
- Evaluate whether a fixed small corpus should bypass retrieval and use cached or loaded context instead.
Context Engineering
Cache-augmented generation is a generation approach that preloads relevant knowledge and reuses its processed context across multiple interactions.
- How it is used
- Uses preloaded external knowledge to reduce repeated retrieval and latency across interactions.
- How deep it goes
- Explains the KV-cache, prompt-caching, and application-state layers, including retention decisions and cache-miss behavior.
- Examples
- Illustrates external knowledge loaded as a cached prompt prefix and reused by subsequent user queries.
- What you can do after
- Evaluate CAG architecture by distinguishing cache layers and considering retention, invalidation, and cache misses.
Read how Context Engineering explains cache-augmented generation →
What to know first
- context windowA context window is a bounded span of sequence elements or contextual information that a model or processing operation can consider together.
Related concepts
- context injectioncontrasts with · Context injection is the technique of adding externally supplied information to a model’s input so it can inform the model’s generated response.
- latencyaddresses · Latency is the elapsed time between initiating an operation or request and receiving its corresponding result or observable response.
- key-value cachecontrasts with · A key-value cache stores attention keys and values computed for earlier tokens so they can be reused during subsequent autoregressive generation.
- prompt cachinguses · Prompt caching stores and reuses previously processed prompt content or computations to reduce repeated processing, cost, or latency.
- generationkind of · Generation is the production of an output by a computational model from supplied input or context, often using patterns learned from data.
Questions about cache-augmented generation
What is cache-augmented generation?
A generation technique that reuses cached information, context, or computation to reduce repeated retrieval or inference costs.
Which Manning books explain cache-augmented generation?
Retrieval Augmented Generation, The Foundational Ideas and Context Engineering.
What should I know before learning cache-augmented generation?
Manning books that teach cache-augmented generation usually introduce context window first.
What is the difference between cache-augmented generation and context injection?
A generation technique that reuses cached information, context, or computation to reduce repeated retrieval or inference costs. Context injection is the technique of adding externally supplied information to a model’s input so it can inform the model’s generated response.
What is cache-augmented generation a kind of?
Cache-augmented generation is a kind of generation. Generation is the production of an output by a computational model from supplied input or context, often using patterns learned from data.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.

