A value vector carries the meaning or substance that a token contributes to an attention computation. Query and Key help decide relevance; Value supplies the content that gets combined. section 11.18 section 11.21
Each token embedding is linearly projected into a Value vector. After attention weights measure influence, those weights scale the vectors, and their sum becomes a context-rich output. section 11.18 section 11.21
Blending token contributions
For token 0, the Value vector is [0.0150, 1.9940, 1.0050, 1.0040]. Its attention weight is 0.0632, producing [0.0009, 0.1261, 0.0635, 0.0635]. Repeating this for other tokens and adding the results gives token 0’s context vector: [0.2283, 2.1990, 0.4526, 1.9747].
Figure 11.34 An analogy of the purposes of Query, Key, and Value
Use the ingredient analogy to distinguish Value as the actual contribution from Query’s request and Key’s descriptive label. section 11.18
