A key vector represents a token for comparison against a query. Dot-product scores are softmax-normalized into weights that determine how much each token’s value contributes to the query token’s enriched embedding. section 3.1.2
In grouped-query attention, query heads within a group share key and value vectors rather than each using an independent set. section 3.2.1
Disambiguating “spring”
A query for “spring” compares against the keys for all tokens. The resulting weights let context such as “water” and “mountain,” or “flowers” and “season,” shape its representation. section 3.1.2
Figure 3.4 Attention mechanism for the token "spring." The Query (Q) from "spring" compares with the Keys (K) of all tokens through dot product operations. The resulting scores, after softmax normalization, determine what fraction of each Value (V) contributes to create the enriched embedding.
Shows a query comparing with all token keys and using the normalized scores to weight their values. section 3.1.2














