A context matrix collects attention outputs for all sequence positions. Each row is a context vector: a weighted sum of value vectors that represents one token in context. If A is the attention-weight matrix and V is the value matrix, the result is C = AV. section 2.2.1 section 3.3.3
To construct A, multiply queries by transposed keys, scale the scores, apply a causal mask, and use softmax row by row. The mask sets future-position scores to negative infinity before softmax, giving those positions zero weight. Each context vector can therefore incorporate its own value and values from earlier positions, but not future values. section 2.2.1 section 2.3.2
Four-token attention
For “The next day is,” an input matrix of shape (4, 8) produces Q, K, and V matrices of shape (4, 4). QKᵀ and the resulting attention weights each have shape (4, 4); multiplying the weights by V gives a (4, 4) context matrix. The row for “is” combines its value vector with those of the preceding tokens. section 2.3.2 section 2.2.1
Figure 2.6 The attention weights are multiplied by the value matrix to produce the final context matrix. Each row is now a context-aware representation of the original token.

Follow the multiplication from attention weights and values to one context-aware row per token. section 2.2.1
In multi-head attention, each head computes its own context matrix using independent projections. The head outputs are concatenated and passed through a final projection. Subsequent transformer processing includes a feed-forward network, which works on each token independently, as well as normalization and residual connections; later output projections produce vocabulary logits. section 2.2.1 section 2.2.2
During autoregressive generation, a full recalculation after appending a token repeats work for earlier positions. Multi-Head Latent Attention instead uses a cached latent matrix for both attention scores and the value-side calculation. In its rearrangement, the value up-projection can be combined with the final output projection. section 2.3.2 section 3.3.3
