A value is an encoder hidden state that carries information into an attention output. Cross-attention combines these values according to attention weights produced from a decoder query and encoder keys. section 12.3.1
The attention computation proceeds as follows:
- Compare the decoder query with encoder keys to produce raw attention scores. section 12.3.1
- Normalize the scores with softmax to obtain attention weights that sum to 1. section 12.3.1
- Compute a weighted sum of the value vectors to produce a context vector for the current prediction step. section 12.3.1
Figure 12.8 The mechanics of cross-attention. A decoder state acts as a query (qj) and compares itself against all encoder states (keys, K). These comparisons generate attention weights (α), which are used to compute a weighted sum of the encoder states (values, V), creating a context vector (cj) specific to the current prediction step.

Trace how a decoder state becomes a query, attends to encoder states, and combines their values into a prediction-specific context vector. section 12.3.1














