A context vector summarizes the information selected for one output step. In cross-attention, a decoder query compares with encoder states, produces attention weights, and combines their value representations into a weighted sum. section 12.3.1
For protein sequences, attention replaces the bottleneck of forcing hundreds of amino acids into one final recurrent hidden state. Each output can draw on the encoder states most relevant to its prediction. section 12.3.1
Figure 12.8 The mechanics of cross-attention. A decoder state acts as a query (qj) and compares itself against all encoder states (keys, K). These comparisons generate attention weights (α), which are used to compute a weighted sum of the encoder states (values, V), creating a context vector (cj) specific to the current prediction step.

Trace how decoder queries, encoder keys, attention weights, and values produce an output-specific context vector. section 12.3.1


