A key vector is an encoder hidden state that a decoder compares with its current query to estimate relevance. The resulting attention weights determine how strongly the associated value contributes to the context vector. In protein structure prediction, keys let each output position retrieve information from nearby or distant input positions instead of relying on one final bottleneck vector. section 12.3.1
Within cross-attention, the three representations have distinct jobs:
- Query: represents the decoder's current focus and information need. section 12.3.1
- Key: represents each encoder position available for comparison. section 12.3.1
- Value: supplies the encoder information combined into the context vector after weighting. section 12.3.1
Figure 12.8 The mechanics of cross-attention. A decoder state acts as a query (qj) and compares itself against all encoder states (keys, K). These comparisons generate attention weights (α), which are used to compute a weighted sum of the encoder states (values, V), creating a context vector (cj) specific to the current prediction step.

Trace how a decoder query compares with encoder keys, produces attention weights, and combines values into a prediction-specific context vector. section 12.3.1














