A query (Q) is a vector generated from a token’s embedding to search for contextual information. It is compared with the keys (K) of tokens in the sequence; the resulting compatibility scores determine how much of each token’s value (V) contributes to the query token’s enriched embedding. section 3.1.2
Figure 3.4 Attention mechanism for the token "spring." The Query (Q) from "spring" compares with the Keys (K) of all tokens through dot product operations. The resulting scores, after softmax normalization, determine what fraction of each Value (V) contributes to create the enriched embedding.
The diagram traces how a query is compared with keys and how the resulting scores weight values. section 3.1.2
The token “spring”
In “Fresh water flows from the mountain spring daily,” the query for “spring” searches the other tokens for clues about its context. A high dot product with the key for “mountain” indicates strong alignment; after softmax normalization, the scores determine each value’s contribution. section 3.1.2













