PagedAttention
PagedAttention is an attention-memory technique that stores key-value cache blocks in independently managed, potentially noncontiguous memory pages.
Transformers in Action
Paged attention is an attention-memory technique that partitions a key-value cache into blocks managed like virtual-memory pages instead of requiring one contiguous allocation.
- How it is used
- Uses PagedAttention to manage KV-cache memory for efficient LLM deployment, especially with long sequences and batched requests.
- How deep it goes
- Explains block-based cache management through a virtual-memory analogy, without detailing block mapping or implementation.
- Examples
- Demonstrates cached versus uncached generation and serves Gemma 3 through vLLM with a chat-completion request.
- What you can do after
- Apply it via vLLM by serving Gemma 3 and sending a chat-completion request.
Rearchitecting LLMs
PagedAttention is an attention-memory management technique that stores key-value cache blocks in pages to improve memory utilization during inference.
- How it is used
- Manages KV-cache memory during vLLM inference by preallocating VRAM for KV blocks.
- How deep it goes
- Explains preallocation and capacity effects, and notes uniform-page constraints, but leaves paging mechanics largely implicit.
- Examples
- Compares dynamic and preallocated cache strategies, then checks predicted cache savings after attention pruning.
- What you can do after
- Evaluate how weight or KV-cache quantization changes token capacity, and verify pruning savings on the target engine.
Domain-Specific Small Language Models
An attention algorithm that stores KV-cache blocks noncontiguously to reduce fragmentation and improve memory flexibility during LLM serving.
- How it is used
- Improves KV-cache memory management for LLM serving by reducing fragmentation.
- How deep it goes
- Explains noncontiguous KV-cache blocks and fragmentation, without detailing the algorithm’s implementation.
- What you can do after
- Understand how noncontiguous KV-cache storage can reduce fragmentation in LLM serving.
Read how Domain-Specific Small Language Models explains PagedAttention →
What to know first
- key-value cacheA key-value cache stores attention keys and values computed for earlier tokens so they can be reused during subsequent autoregressive generation.
- throughputThroughput is the rate at which a system processes or completes work, data, or requests over a specified period.
- attentionAttention is a neural-network mechanism that computes input-dependent relevance weights to combine information from representations for a context-dependent computation.
- meanThe mean is a measure of central location equal to the arithmetic average of numerical observations, or the probability-weighted average of a random variable’s possible values.
Related concepts
- vLLMimplemented by · vLLM is an open-source inference and serving engine for large language models, designed for high-throughput and memory-efficient execution.
- key-value cacheuses · A key-value cache stores attention keys and values computed for earlier tokens so they can be reused during subsequent autoregressive generation.
- virtual memoryuses · Virtual memory is a memory-management abstraction that provides processes with virtual address spaces mapped to physical memory and, when needed, secondary storage.
- memory fragmentationaddresses · Memory fragmentation is the division of available memory into separated or inefficiently sized regions that reduce its effective usability or allocation efficiency.
- cachinguses · Caching is the temporary storage of data or computed results for reuse, reducing the work or delay of retrieving or computing them again.
Questions about PagedAttention
What is PagedAttention?
PagedAttention is an attention-memory technique that stores key-value cache blocks in independently managed, potentially noncontiguous memory pages.
Which Manning books explain PagedAttention?
Transformers in Action, Rearchitecting LLMs and Domain-Specific Small Language Models.
What should I know before learning PagedAttention?
Manning books that teach PagedAttention usually introduce key-value cache, throughput, attention and mean first.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.


