A value vector is the information a token offers when attention builds a context-aware representation. Attention first decides how relevant each token is, then uses those weights to combine their values. section 2.1.3
The attention process works in a short sequence:
- Each token supplies a query, a key, and a value: what it seeks, what it offers, and what information it carries. section 2.1.3
- The query compares with other tokens' keys using dot products, producing relevance scores. section 2.1.3
- Softmax turns the scores into weights, and the model multiplies each value by its weight before summing the results. section 2.1.3
Resolving “it”
In “The performance was unbelievable—the crowd couldn't believe it,” the query for “it” gives “performance” the strongest weight, “crowd” a smaller one, and “couldn't” almost none. The weighted value sum enriches “it” with mostly information from “performance.” section 2.1.3
Figure 2.3 When processing "it," the model assigns an attention weight to every other token in the sentence. "Performance" receives the highest weight by far (0.68 out of 1.0)which is how the model infers what "it" refers to.

Use the attention figure to see how weights assigned to surrounding tokens determine which value information contributes most strongly. section 2.1.3














