When a model processes one token, its key answers a simple question: “What information do I have to offer?” Other tokens compare their queries against that key to decide whether it is relevant. section 2.1.3
The model compares a query with every token’s key using a dot product. Higher matches receive larger softmax weights, allowing the matching token’s value to contribute more to the updated representation. section 2.1.3
Resolving “it”
In “The performance was unbelievable—the crowd couldn’t believe it,” the query for “it” matches “performance” strongly, “crowd” weakly, and “couldn’t” hardly at all. The key for “performance” therefore helps make that token the most relevant source of information. section 2.1.3
Figure 2.3 When processing "it," the model assigns an attention weight to every other token in the sentence. "Performance" receives the highest weight by far (0.68 out of 1.0)which is how the model infers what "it" refers to.

Use the attention-weight figure to see how one token’s query assigns different relevance weights to the keys of the surrounding tokens. section 2.1.3














