Cross-attention
Cross-attention is an attention mechanism in which queries from one sequence or representation attend to keys and values from another.
Retrieval Augmented Generation, The Foundational Ideas
Cross-attention is an attention mechanism in which one sequence selects information from the representations of another sequence.
- How it is used
- Enables a generator to select and combine retrieved-document evidence while producing an answer.
- How deep it goes
- Explains FiD’s independent encoding, tensor reshaping, and decoder access at each generation step, without deriving attention mathematics.
- Examples
- Uses multi-passage historical reasoning and a Transformers decoding example that reshapes hidden states and attention masks.
- What you can do after
- Apply it via Transformers to expose separately encoded passages to one decoder, while accounting for training and generation latency.
Read how Retrieval Augmented Generation, The Foundational Ideas explains cross-attention →
Build AI Drug Discovery Pipelines
Cross-attention is an attention mechanism in which queries from one sequence or representation attend to keys and values derived from another.
- How it is used
- Introduces cross-attention as a way for a decoder to access input representations without a single-vector bottleneck.
- How deep it goes
- Explains query-to-key comparisons, attention weights, and the weighted context vector for each decoding step.
- Examples
- Uses a 500-residue protein and a possible disulfide bond between cysteines at positions 45 and 312.
- What you can do after
- Understand how a decoder can draw on different input positions for each output.
Read how Build AI Drug Discovery Pipelines explains cross-attention →
Build a Text-to-Image Generator (from Scratch)
An attention mechanism that computes attention between decoder representations and representations from an encoded conditioning input.
- How it is used
- Connects decoder states to encoded source inputs during translation and image captioning.
- How deep it goes
- Specifies decoder-layer placement, query/key/value inputs, and a concrete call in code; does not elaborate the attention calculation here.
- Examples
- Uses German-to-English translation, a skiing-image caption, and a decoder-layer implementation with separate source and target masks.
- What you can do after
- Build it from scratch as a decoder sublayer that attends to encoder output for translation or image captioning.
Read how Build a Text-to-Image Generator (from Scratch) explains cross-attention →
Sutskever's List
An attention mechanism in which queries come from one sequence or representation and keys and values come from another.
- How it is used
- Aligns generated output with encoded input sequences inside an encoder-decoder generation pipeline.
- How deep it goes
- Defines Q/K/V separation and contextual alignment, but stops short of deriving attention scores or detailing implementation equations.
- Examples
- Uses translation alignment and a Transformer encoder-decoder schematic to illustrate selective source-context retrieval.
- What you can do after
- Apply it via encoder-decoder attention to guide token generation from encoded input representations.
Build an Advanced RAG Application (From Scratch)
An attention mechanism in which queries from one sequence or representation attend to keys and values from another.
- How it is used
- Guides decoder generation by connecting output-token prediction with relevant encoded input information.
- How deep it goes
- Explains the decoder-layer role and generation behavior, but does not detail attention equations or implementation internals.
- Examples
- Uses machine translation, text summarization, and text generation as application examples.
- What you can do after
- Evaluate/understand how decoder models use encoded inputs during autoregressive text generation.
Read how Build an Advanced RAG Application (From Scratch) explains cross-attention →
Learn Generative AI with PyTorch
An attention operation where queries come from one sequence or representation and keys/values come from another, enabling information transfer between them.
- How it is used
- Connects decoder states to encoder representations so Transformers can generate translations conditioned on an input sequence.
- How deep it goes
- Traces queries, keys, scaled scores, softmax weights, decoder ordering, masks, residual connections, and layer normalization through code and diagrams.
- Examples
- Uses French translation tokens, encoder memory, a five-by-five target mask, attention-weight calculations, and a PyTorch DecoderLayer implementation.
- What you can do after
- Build it from scratch by assigning decoder queries and encoder keys and values within a Transformer decoder layer.
Read how Learn Generative AI with PyTorch explains cross-attention →
Building Specialized AI Systems
Cross-attention lets decoder positions attend to representations produced by an encoder.
- How it is used
- Connects decoder generation to encoder representations in encoder–decoder sequence transduction.
- How deep it goes
- Explains its role within decoder blocks and contrasts it with causal self-attention, without detailing projections or training procedures.
- Examples
- Illustrates cross-attention with encoder–decoder language translation and a Transformer architecture diagram.
- What you can do after
- Evaluate how encoder–decoder Transformers combine source representations with causal target generation.
Read how Building Specialized AI Systems explains cross-attention →
Machine Learning Algorithms in Depth
Cross-attention computes attention with queries from one sequence or representation source and keys and values from another.
- How it is used
- Associates decoder output positions with encoder representations during sequence generation and machine translation.
- How deep it goes
- Defines cross-attention through differing query, key, and value sources and explains its role, but does not derive numerical computation.
- Examples
- Uses neural machine translation, with decoder target tokens querying encoded input tokens.
- What you can do after
- Apply cross-attention to relate each generated target word to relevant words in an encoder-decoder transformer.
Read how Machine Learning Algorithms in Depth explains cross-attention →
Also covered in
- Transfer Learning for Natural Language Processing
- Machines that ThinkIn early access (MEAP)
- Deep Learning with PyTorch, Second Edition
- Crack Any Codebase with AIIn early access (MEAP)
- Deep Learning with Python, Third Edition
- Deep Learning with R, Third Edition
- Grokking AI Algorithms, Second Edition
What to know first
- encoderAn encoder is a component or process that transforms input into an encoded or internal representation for subsequent processing, transmission, or decoding.
- attentionAttention is a neural-network mechanism that computes input-dependent relevance weights to combine information from representations for a context-dependent computation.
- encoder-decoder architectureAn encoder-decoder architecture is a neural model design in which an encoder transforms input into a representation and a decoder uses that representation to produce output, often mapping one sequence to another.
- softmaxSoftmax is a function that exponentiates each value in a score vector and normalizes the results to produce positive values summing to one, forming a probability distribution.
Related concepts
- attentioncontrasts with · Attention is a neural-network mechanism that computes input-dependent relevance weights to combine information from representations for a context-dependent computation.
- encoderuses · An encoder is a component or process that transforms input into an encoded or internal representation for subsequent processing, transmission, or decoding.
- self-attentioncontrasts with · Self-attention is an attention mechanism that relates positions within a sequence, allowing each position to incorporate information from other positions in that sequence.
- decoder layerused by · A decoder layer is a neural-network component that transforms representations to produce an output sequence, often using self-attention and, in encoder–decoder models, information from an encoder.
- decoding strategyused by · A decoding strategy is a procedure for converting an encoded or internal representation into a target output, including selecting tokens during sequence generation.
- fusion-in-decoderused by · Fusion-in-decoder is a retrieval-generation method that independently encodes multiple passages and fuses their representations in the decoder to generate an output.
Questions about cross-attention
What is cross-attention?
Cross-attention is an attention mechanism in which queries from one sequence or representation attend to keys and values from another.
Which Manning books explain cross-attention?
Retrieval Augmented Generation, The Foundational Ideas, Build AI Drug Discovery Pipelines, Build a Text-to-Image Generator (from Scratch), Sutskever's List, Build an Advanced RAG Application (From Scratch), Learn Generative AI with PyTorch, Building Specialized AI Systems and Machine Learning Algorithms in Depth.
What should I know before learning cross-attention?
Manning books that teach cross-attention usually introduce encoder, attention, encoder-decoder architecture and softmax first.
What is the difference between cross-attention and attention?
Cross-attention is an attention mechanism in which queries from one sequence or representation attend to keys and values from another. Attention is a neural-network mechanism that computes input-dependent relevance weights to combine information from representations for a context-dependent computation.
What is cross-attention also called?
Cross-attention is also called encoder-decoder attention.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.







