A query is a learned vector that represents what a token seeks from its context. In attention, it is compared with other tokens’ keys to assess relevance; the corresponding values carry information that can contribute to the token’s updated representation. section 4.1.3
The query works with two other learned projections:
- A key provides characteristics used to judge whether its token is relevant to the query. section 4.1.3
- A value carries information that can be added to the representation when its token receives attention. section 4.1.3
Interpreting “rises”
For “rises” in “The sun rises in the east,” its query can align more strongly with the key for “sun” than with the key for “the.” The comparison produces relevance scores; values, rather than queries, carry the information used to update the representation. section 4.1.3
Figure 4.4 The query for “rises” is compared with the keys of other tokens to produce relevance scores. A stronger match with “sun” gives it a higher score than “the,” indicating that it is more useful for interpreting “rises.”

The diagram shows how the query for “rises” is compared with other tokens’ keys to produce relevance scores. section 4.1.3
Queries also connect encoder and decoder representations in cross-attention. At each decoding step, the decoder’s current representation supplies the query, while the encoder’s output supplies keys and values. This lets the decoder retrieve relevant input information without directly accessing future tokens in its own sequence. section 4.1.7













