How can a model notice several kinds of relationships in the same sentence? Multi-head attention runs multiple attention processes in parallel, then combines their results. Different heads may learn to focus on grammar, meaning, or distant connections. section 2.1.3
Several readers, one sentence
When processing “it,” one head might connect it with “performance,” while others track grammatical structure or relationships between distant words. The heads’ observations combine into a richer, context-aware representation. These roles emerge through training rather than being assigned by designers. section 2.1.3
Figure 2.3 When processing "it," the model assigns an attention weight to every other token in the sentence. "Performance" receives the highest weight by far (0.68 out of 1.0)which is how the model infers what "it" refers to.

The figure shows attention weights linking “it” to other tokens, with “performance” receiving the strongest weight. section 2.1.3
























