Multi-query attention (MQA) reduces key-value cache memory during token-by-token generation: all query heads share one set of keys and values. This extreme sharing saves cache space, but is associated with quality compromises relative to grouped-query attention. section 3.1.2
Figure 3.5 Attention mechanisms and memory optimization strategies. Multi-Head Attention (MHA) maintains independent K,V pairs for each Q head — one pair per processed token, forming a growing cache. Grouped-Query Attention (GQA) groups Q heads to share K,V pairs (here 6 Q heads in 3 groups, each sharing one K,V pair per token). Multi-Query Attention (MQA) compresses to the extreme: all Q heads share a single K,V pair per token.
Compare how multi-head, grouped-query, and multi-query attention assign key-value pairs to query heads. section 3.1.2
Cache capacity comparison
For a six-layer, 12-head example, MQA fits 96 tokens in a 1,152-vector cache budget, compared with 8 tokens for multi-head attention. section 3.1.2

