liveBook · Manning

Encoder

As explained in Retrieval Augmented Generation, The Foundational Ideas

An encoder is a model component that transforms an input sequence into a numerical representation.

A retrieval system can return near-duplicates that repeat a question’s wording while missing the passage that answers it. An encoder transforms text into numerical vectors; changing its weights changes the vector space, so nearest-neighbor search can rank useful passages above lookalikes. section 3.0

In Fusion-in-Decoder, the encoder processes each question-and-passage input independently and in parallel. Self-attention stays inside each sequence, producing N representations with shape [N, sequence_length, hidden]. A reshape lays them end to end; the decoder then uses cross-attention across the resulting sequence to synthesize an answer. section 5.2

The encoder’s role differs across retrieval-augmented designs:

  • In a trained retriever, encoder weights move questions and passages in vector space so instrumental usefulness can influence ranking. section 3.0
  • In FiD, each retrieved passage receives a separate question-plus-passage encoding, while the decoder performs the cross-passage fusion. section 5.2

Encoding a retrieved passage

For the question “Where was T5 developed?”, one encoder input combines question:, title:, and context: segments with a single retrieved passage. Repeating this structure for N passages yields independent representations for decoder conditioning. section 5.2

Figure 5.2 The FiD architecture: Independent Encoding, Fused Decoding.

Diagram the separation between independent passage encoding and fused decoder attention. section 5.2

Table 5.2 Comparison between Retrieval-Extractive, Closed-Book Generative, and FiD
Aspect
Retrieval-Extractive (DrQA)
Closed-book Generative (T5)
Fusion-in-Decoder (hybrid)
Core idea
Retrieve candidate passages, then extract a text span from each passage with a reader.
Store knowledge in model parameters; generate an answer directly from the model (no retrieval at inference).
Retrieve multiple passages, encode them, then use a generative decoder that conditions on all encodings to synthesize an answer.
Knowledge source
External corpus (Wikipedia, web) queried at inference time.
Implicitly, in the model weights learned during pretraining.
External corpus + generative decoder — external knowledge retrieved at inference, fused by decoder.
Reader/Architecture
Separate retriever + extractive reader (span prediction per passage).
Encoder-decoder Transformer trained in text-to-text fashion (generative).
Retriever + encoder for each passage; decoder attends over concatenated/encoded passages.
Answer format
Literal span from a retrieved passage (extractive).
Abstractive, free-form text (can paraphrase or synthesize).
Abstractive, but grounded in retrieved passages; can combine evidence across passages.

Compare encoder-based retrieval-generation designs with extractive readers and closed-book generation. section 5.2

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Explains vector-space movement, independent self-attention, tensor shape, reshaping, and decoder cross-attention.
Examples
Uses question-title-context inputs, multiple retrieved passages, vector ranking, and the FiD tensor shape [N, sequence_length, hidden].
What you can do after
Evaluate/understand how encoding changes retrieval rankings and how FiD exposes multiple passage representations to a decoder.
How it connects
Connects encoders to retrieval, nearest-neighbor search, self-attention, decoders, cross-attention, and encoder-decoder Transformers.

Also explained in

Compare how 28 books explain encoder →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.