In attention, a value is a representation of information that can contribute to another token’s contextual representation. Attention weights determine how much each value contributes: the output is a weighted combination of the values, rather than a selection based on a fixed grammatical rule. section 12.2.1
Queries and keys determine attention scores; the resulting weights are then applied to values. The distinction matters: scores describe how strongly tokens relate, while values supply the information being combined. In a simplified calculation, the query, key, and value are each set equal to the token embedding: q = k = v = e. section 12.2.1
Combining values for a pronoun
Suppose the values for “trophy” and “box” are [1, 0] and [0, 1], and the attention weights for “it” are 0.39 and 0.61. Its resulting representation is z_it = 0.39[1, 0] + 0.61[0, 1] = [0.39, 0.61]. The larger weight gives the box value more influence. These two-dimensional numbers are illustrative, not outputs from a trained model. section 12.2.1
The contextual representation contributes to predicting a probability distribution over possible next tokens. Real transformers use larger representations, multiple attention heads, feed-forward networks, and stacked layers, so the example isolates only part of that process. section 12.2.1














