liveBook · Manning

Chapter 5: Fusion-in-Decoder for multi-document processing

What you will learn in chapter 5 of Retrieval Augmented Generation, The Foundational Ideas

The introduction explains why answers spread across retrieved documents are difficult for conventional RAG and presents FiD’s separation of independent encoding and fused decoding as a scalable way to synthesize them.

After this chapter you will be able to

What it builds on

Section by section

Introduction

The introduction explains why answers spread across retrieved documents are difficult for conventional RAG and presents FiD’s separation of independent encoding and fused decoding as a scalable way to synthesize them.

Key terms: top-k documents, FP7 (Incomplete), FP5 (Not Extracted), Fusion-in-Decoder, independent encoding, fused decoding, cross-attention, encoder, decoder, precision

5.1 The challenges of multiple document contexts

This section explains why extractive readers, closed-book generators, and long-context prompting struggle with multi-document questions, motivating FiD’s approach to scalable evidence synthesis.

Key terms: retrieval-extractive pipeline, DrQA, answer spans, sparse retriever, recall, precision, retriever, reader, distractors, T5

5.2 Independent encoding, fused decoding

This section explains how FiD encodes retrieved passages separately, concatenates their token representations, and lets a decoder synthesize an answer from all of them.

Key terms: independent encoding, question:, title:, context:, self-attention, final hidden-state representations, fusion, concatenation, reshape, sequence length dimension

5.3 Scaling to hundreds of documents

This section shows that FiD's answer accuracy improves as it receives more passages, explains how its decoder combines evidence across them, and clarifies that this behavior requires training.

Key terms: FiD, retrieved passages, Exact Match, TriviaQA, Natural Questions, extractive readers, Exact Match (EM), F1, canonical one, decoder

5.3.1 Scaling training

This section explains how FiD reduces the cost of training with many passages and how its ability to use them changes retrieval and passage-count decisions.

Key terms: broad adaptation, specialization, fine-tuning, passages, retriever, reader, precision, recall, FiD decoder, re-ranker

5.4 Implementing FiD with transformer models

This section implements FiD with a T5-style encoder-decoder model, then shows how to generate answers, fine-tune on reference answers, and diagnose a model-capability failure.

Key terms: encoder, decoder, cross-attention, question, ctxs, passages, encode_passages, encoder_hidden_states, encoder_outputs, BaseModelOutput

5.5 Case study: Finding the FiD sweet spot

The case study contrasts FiD’s success at selecting one answer-bearing passage with an untrained reader’s failure to synthesize an answer across passages.

Key terms: independent encoding, distractor documents, encoded representations, multi-document synthesis, decoder, fine-tuning

5.6 Applications and performance characteristics

This section explains where FiD is useful, which RAG failures it can mitigate, and the synthesis and inference costs of its design.

Key terms: multi-document synthesis, open-domain question answering, Multi-document Summarization, Fact Checking, Informed Dialogue Systems, Listwise Reranking, FP7 (Incomplete), FP5 (Not Extracted), decoder, independent encoding

5.6.1 FiD's evolution

This section explains how FiD variants filter, reweight, and focus retrieved evidence or reduce decoder memory costs, while leaving failures in initial retrieval unresolved.

Key terms: KG-FiD, Graph Neural Network (GNN), reranking, pruning, FiDO, cross-attention sparsification, parameter reallocation, memory bandwidth bottleneck, RFiD, rationale signal

5.7 Summary

Where this is used later

Concepts explained in this chapter

← Chapter 4: Retrieval-augmented generation for knowledge tasks Chapter 6: Atlas: Few-shot learning with retrieval augmentation →

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.