The absorption trick makes Multi-Head Latent Attention (MLA) scores computable directly from a compressed latent KV cache. It pre-combines fixed query and key projection weights, avoiding the need to cache or reconstruct full keys for the score calculation. section 3.3.2 section 3.3
Start with Q = XWq and K = (XWdkv)Wuk, where Wdkv down-projects the input and Wuk up-projects the latent representation into keys. By associativity, QKᵀ = X(WqWukᵀ)(XWdkv)ᵀ. The product WqWukᵀ is fixed during inference, so it can be pre-combined. The historical term XWdkv is the latent KV cache, cKV. section 3.3.2
Figure 3.4 The mathematical rearrangement of the attention score calculation in MLA.

Follow the rearrangement from full query and key projections to a pre-combined projection and the latent cache. section 3.3.2
Pre-combining weights does not eliminate runtime projection work. It changes which representation is used: the current input produces an absorbed query, which is multiplied by the transpose of the updated latent cache to obtain content scores. section 3.3.2 section 3.14.4
Updating the cache for one token
Suppose the historical latent cache has shape (4, 4). A new token embedding of shape (1, 8) is multiplied by Wdkv of shape (8, 4), producing a (1, 4) latent vector. Appending it gives a (5, 4) cache; the absorbed content query then multiplies its transpose to produce content scores. section 3.14.4
Standard Rotary Positional Encoding (RoPE) obstructs this rearrangement: its position-dependent rotations intervene between the weights that must be combined. Decoupled RoPE separates a cache-efficient content path from a position path; their scores are added to form the final attention scores. section 3.13 section 3.18
