liveBook · Manning

Multi-query attention

As explained in Rearchitecting LLMs

Multi-query attention is an attention architecture in which multiple query heads share a single set of key and value vectors.

Multi-query attention (MQA) reduces key-value cache memory during token-by-token generation: all query heads share one set of keys and values. This extreme sharing saves cache space, but is associated with quality compromises relative to grouped-query attention. section 3.1.2

Figure 3.5 Attention mechanisms and memory optimization strategies. Multi-Head Attention (MHA) maintains independent K,V pairs for each Q head — one pair per processed token, forming a growing cache. Grouped-Query Attention (GQA) groups Q heads to share K,V pairs (here 6 Q heads in 3 groups, each sharing one K,V pair per token). Multi-Query Attention (MQA) compresses to the extreme: all Q heads share a single K,V pair per token.

Compare how multi-head, grouped-query, and multi-query attention assign key-value pairs to query heads. section 3.1.2

Cache capacity comparison

For a six-layer, 12-head example, MQA fits 96 tokens in a 1,152-vector cache budget, compared with 8 tokens for multi-head attention. section 3.1.2

How Rearchitecting LLMs covers it

How deep it goes
Explains shared key-value storage and illustrates cache-memory trade-offs against multi-head and grouped-query attention.
Examples
Compares cache capacity for a six-layer, 12-head example using a 1,152-vector budget.
What you can do after
Evaluate MQA's cache-memory advantage against its stated quality compromise relative to grouped-query attention.
How it connects
Connects multi-head attention, grouped-query attention, query heads, and key-value caching.

Also explained in

Compare how 2 books explain multi-query attention →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Rearchitecting LLMs. The section links point to the parts of the book it draws on.