A key is a learned vector derived from a token’s representation. It describes characteristics that help a query assess whether that token is relevant. Unlike a key, a value carries information that can be added to another token’s representation. section 4.1.3
An attention layer compares a query with keys using a dot product: stronger alignment produces a higher relevance score. Keys can come from the same sequence as the query in self-attention, or from encoder representations in cross-attention. section 4.1.3 section 4.1.7
Matching “rises”
For “rises” in “The sun rises in the east,” its query can match the key for “sun” more strongly than the key for “the,” giving “sun” a higher relevance score. section 4.1.3
Figure 4.3 A single token representation is transformed by three learned projections into a query, key, and value, which serve different roles in attention. The query represents what the token looks for in context, the key what makes it potentially relevant to other tokens, and the value the information it can contribute.

The diagram distinguishes the key’s relevance-matching role from the query’s search role and the value’s information-carrying role. section 4.1.3














