A dot product turns two vectors into one scalar: multiply corresponding components, then add the results. In attention, that scalar measures how closely a query matches a key. section 2.1.3
Finding what “it” refers to
When processing “The performance was unbelievable—the crowd couldn't believe it,” the query for “it” is compared with every token's key. A strong dot-product score for “performance” signals high relevance; weaker scores for “crowd” and “couldn't” signal less relevance. Softmax then converts these scores into attention weights. section 2.1.3
Figure 2.3 When processing "it," the model assigns an attention weight to every other token in the sentence. "Performance" receives the highest weight by far (0.68 out of 1.0)which is how the model infers what "it" refers to.

See how attention distributes weights across the sentence, giving the strongest weight to “performance” when processing “it.” section 2.1.3


















