liveBook · Manning

Multi-query attention

Multi-query attention is an attention architecture in which multiple query heads share key and value projections.

How 2 Manning books explain it

Build a DeepSeek Model (From Scratch)

Multi-query attention uses multiple query heads that share a smaller set of key and value projections to reduce attention-memory requirements.

How it is used
Reduces KV cache memory during autoregressive inference by sharing key and value projections across query heads.
How deep it goes
Derives cache savings, explains the expressivity trade-off, and implements a PyTorch attention layer.
Examples
Uses a counterfactual GPT-3 cache estimate and a controlled four-architecture validation-loss and memory comparison.
What you can do after
Build it from scratch as a PyTorch layer with shared key and value projections.

Read how Build a DeepSeek Model (From Scratch) explains multi-query attention →

Rearchitecting LLMs

Multi-query attention is an attention architecture in which multiple query heads share a single set of key and value vectors.

How it is used
Reduces key-value cache memory during token-by-token generation by sharing one key-value set across query heads.
How deep it goes
Explains shared key-value storage and illustrates cache-memory trade-offs against multi-head and grouped-query attention.
Examples
Compares cache capacity for a six-layer, 12-head example using a 1,152-vector budget.
What you can do after
Evaluate MQA's cache-memory advantage against its stated quality compromise relative to grouped-query attention.

Read how Rearchitecting LLMs explains multi-query attention →

Related concepts

Questions about multi-query attention

What is multi-query attention?

Multi-query attention is an attention architecture in which multiple query heads share key and value projections.

Which Manning books explain multi-query attention?

Build a DeepSeek Model (From Scratch) and Rearchitecting LLMs.

What is the difference between multi-query attention and multi-head attention?

Multi-query attention is an attention architecture in which multiple query heads share key and value projections. Multi-head attention is an attention mechanism that computes multiple attention operations in parallel using distinct learned projections and combines their outputs.

What is multi-query attention a kind of?

Multi-query attention is a kind of attention. Attention is a neural-network mechanism that computes input-dependent relevance weights to combine information from representations for a context-dependent computation.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.