A query vector is the decoder’s current hidden state: it represents the information currently sought from encoder hidden states during cross-attention. section 12.3.1
Cross-attention uses the query in a short retrieval workflow:
- Compare the query with every encoder hidden state, treated as a key, to produce compatibility scores or logits. section 12.3.1
- Pass the scores through softmax to obtain normalized attention weights over the encoder positions. section 12.3.1
- Use those weights to combine the encoder hidden states, treated as values, into a context vector for the current output. section 12.3.1
Protein-sequence decoding
While generating an output from encoded protein information, each decoding step produces a query that selects the most relevant encoder positions rather than relying on one fixed context vector. section 12.3.1
Figure 12.8 The mechanics of cross-attention. A decoder state acts as a query (qj) and compares itself against all encoder states (keys, K). These comparisons generate attention weights (α), which are used to compute a weighted sum of the encoder states (values, V), creating a context vector (cj) specific to the current prediction step.

The cross-attention diagram traces a decoder query through key comparisons, attention weights, and the resulting context vector. section 12.3.1













