Cross-attention lets one representation select information from another. In an encoder–decoder model, the decoder uses its current state to attend to the encoder’s hidden states, rather than relying on a single vector to summarize the entire input. section 12.3.1
At each output step, the decoder state acts as a query. It is compared with encoder states acting as keys to produce attention weights. Those weights determine how much each encoder state, acting as a value, contributes to a weighted-sum context vector for that step. Different output steps can therefore draw on different input positions. section 12.3.1
Figure 12.8 The mechanics of cross-attention. A decoder state acts as a query (qj) and compares itself against all encoder states (keys, K). These comparisons generate attention weights (α), which are used to compute a weighted sum of the encoder states (values, V), creating a context vector (cj) specific to the current prediction step.

Follow how a decoder query produces weights over encoder states and a context vector for the current prediction. section 12.3.1
Distant residues in a protein
Suppose a protein has 500 amino acids, and the task is to predict whether cysteines at positions 45 and 312 form a disulfide bond. Instead of forcing the final encoder state to hold both contexts, cross-attention can give the decoder access to the hidden states at those positions and elsewhere in the sequence. section 12.3.1
A standard transformer uses cross-attention in its decoder to access the encoder’s sequence representations at each generation step. An encoder-only transformer instead keeps the encoder and discards the decoder; ESM-2 uses this approach to represent protein sequences. section 12.3.2







