When a token such as “it” must draw on surrounding context, a query represents what that token is seeking. In self-attention, the query is compared with keys, which represent how other tokens can be identified as relevant. Their values contain the information they can contribute. section 12.2.1
A query is produced by applying a learned projection matrix, WQ, to a token’s embedding. Separate learned projections produce keys and values, allowing the same token to take different roles in seeking information, signaling relevance, and contributing content. section 12.2.1
A query–key dot product measures vector alignment and supplies a compatibility score. Scores are scaled and passed through softmax to produce attention weights; those weights determine how much each value contributes to the contextual representation. section 12.2.1
Resolving a pronoun
Suppose the query for “it” is [0.2, 0.8], and the keys for “trophy” and “box” are [1, 0] and [0, 1]. Their dot products with the query are 0.2 and 0.8, respectively. After scaling and softmax, the two weights are approximately 0.39 and 0.61, so the box value contributes more to the representation of “it.” This calculation assumes identity projections and uses deliberately simple vectors; the embedding components are not probabilities or necessarily interpretable dimensions in a trained model. section 12.2.1
A single query–key calculation illustrates one contribution to context, not the full behavior of a transformer. Trained models combine many attention heads, feed-forward networks, and stacked layers. section 12.2













