When a language model encounters an ambiguous token, it must determine which surrounding words matter most. Attention solves this by assigning different weights to parts of the input sequence when computing a representation or output. section 13.2.2
In a Transformer-based language model, attention lets the network focus on sequence elements relevant to the current task while assigning less weight to others. This supports higher-level representations beyond predicting each token from the tokens before it. section 13.2.1 section 13.2.2
Stacked attention layers can build progressively broader relationships:
- Early layers capture immediate word-to-word relationships. section 13.2.2
- Intermediate layers connect phrases and treat related spans as broader entities. section 13.2.2
- Later layers represent overarching themes across a sentence or broader text. section 13.2.2
Resolving “leaves”
In “The Labor Day weekend wildfire started in the forest and spread rapidly due to the dry leaves on the ground,” attention connects “leaves” with “dry,” “ground,” and “forest,” favoring the foliage meaning over the verb meaning. section 13.2.2
A higher-level representation is not automatically a causal representation. Attention can help an LLM organize relevant information, but it does not by itself establish causal understanding or make the model’s reasoning verifiable. section 13.4
