liveBook · Manning

Decoder

As explained in Retrieval Augmented Generation, The Foundational Ideas

A decoder is a neural-network component that generates an output sequence from encoded representations and its previously generated tokens.

A decoder generates an output sequence from encoded representations and previously generated tokens. In Fusion-in-Decoder, it attends simultaneously to independently encoded question–passage representations and synthesizes an answer. section 5.2

In multi-document processing, the decoder:

  • Reads the concatenated encoder representations as one sequence of length N × sequence_length. section 5.2
  • Uses cross-attention to access all retrieved passages while generating each answer token. section 5.7
  • Enables synthesis across scattered evidence, while making inference latency the main trade-off because attention spans the full concatenated sequence. section 5.7

Question-and-passage decoding

For “Where was T5 developed?”, independently encode question–title–context inputs for retrieved passages, concatenate their representations, and let the decoder generate a synthesized answer. section 5.2

Figure 5.2 The FiD architecture: Independent Encoding, Fused Decoding.

Use the architecture diagram to trace independent encoding, representation fusion, and decoder attention over all passages. section 5.2

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Explains cross-attention over reshaped encoder outputs and identifies latency from attending to the full concatenated sequence.
Examples
Uses question–title–context inputs and a multi-passage T5-style FiD architecture to illustrate answer generation.
What it assumes
Assumes familiarity with encoded representations, retrieved passages, encoder-decoder models, and cross-attention.
What you can do after
Apply it via Fusion-in-Decoder to synthesize answers from independently encoded retrieved passages.
How it connects
Connects decoders with encoders, cross-attention, passage retrieval, T5, and Fusion-in-Decoder.

Also explained in

Compare how 9 books explain decoder →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.