Cross-attention lets one representation select information from another. In the Transformer decoder, queries come from the partially generated output, while keys and values come from the encoder, aligning generation with the input sequence. section 5.1
Translation alignment
During translation, each decoding step uses its current query to score encoded input representations, producing a context vector that guides the next output token. Unlike masked self-attention, this operation can consult the encoded source sequence. section 5.1
Figure 5.1 A simplified schematic of the Transformer architecture. The model consists of an encoder (left) and a decoder (right), each comprising repeated stacks of sublayers, including multihead attention mechanisms, feed-forward neural networks, and postprocessing steps such as residual connections and layer normalization. Positional encodings (depicted as circled crosses) are added to the input embeddings to explicitly represent token order. The diagram emphasizes the cross-attention operation (dashed box labeled “Cross-attention”), in which the decoder’s queries (Q) interact with the encoder’s keys (K) and values (V). During cross-attention, each decoding step computes attention scores to determine the relevance of encoded input representations, producing a context vector that guides token generation. This selective alignment enables the decoder to integrate contextual information from the encoder, improving the accuracy and coherence of the output sequence.
Use the Transformer schematic to trace decoder queries interacting with encoder keys and values inside the repeated encoder-decoder architecture. section 5.1
