liveBook · Manning

End-to-end learning

As explained in Retrieval Augmented Generation, The Foundational Ideas

End-to-end training jointly optimizes connected system components by propagating learning signals through the complete input-to-output pipeline.

A retriever can find passages that resemble a question yet rank the passage containing the answer below them. End-to-end learning addresses this mismatch by training retrieval and answer production together. The goal is not to change nearest-neighbor search itself: training changes the text encoder so passages useful to the reader move closer to the queries that need them. section 3.0 section 6.8

In a separately assembled pipeline, a fixed retriever selects documents using similarity, whether or not they help the reader answer. Joint training closes that loop: the reader’s performance supplies a learning signal for the retriever. This matters when a passage repeats the question’s terms but another passage supplies the explanation the answer requires. section 3.0 section 6.8

Learning from a missing word

REALM connects a neural retriever to a language-model reader during pre-training. Given “The monarch butterfly migrates to [MASK] for the winter,” the retriever can fetch evidence about migration to Mexico; predicting “Mexico” then rewards useful retrieval. The text supplies its own answer key, without human relevance labels. Masking a name or date makes retrieval valuable, whereas masking a function word may let the reader succeed from grammar alone and give the retriever little signal. section 3.2

Joint learning can use different objectives and incur different costs:

  • REALM jointly pre-trains retriever and reader using masked-token prediction as the signal for retrieval usefulness. section 3.2
  • Atlas uses Perplexity Distillation (PDist): it measures how likely the reader finds the target answer with each retrieved document, then trains retriever scores toward that distribution. Other objectives use decoder attention, marginalize over documents, or measure the damage from removing one; removal is more faithful to multi-document inference but costs an extra forward pass per document. section 6.2.2
  • Canonical RAG jointly fine-tunes retriever and generator and weighs contributions across retrieved documents. Naive RAG instead puts retrieved chunks into an off-the-shelf model’s prompt without joint training, trading potential task-specific gains for lower training and deployment complexity. section 4.7
Table 6.3 Training objectives
Training Objective
Core Mechanism
Computational Cost
Key Insight
Attention Distillation (ADist)
Distills decoder cross-attention scores into the retriever.
Medium
Uses the reader's internal attention as a proxy for document relevance.
EMDR²
Marginalizes over documents to maximize the target output's likelihood.
Medium
Treats documents as latent variables in an end-to-end probabilistic model.
Perplexity Distillation (PDist)
Trains a retriever to predict the LM's perplexity for each document.
Low
A simple, direct, and efficient way to align retriever scores with LM utility.
LOOP
Measures importance by the performance drop when a document is removed.
High
More closely aligns with multi-document inference but is expensive.

Compare four ways to turn reader behavior into retriever supervision, including their relative computational costs. section 6.2.2

Finding the explanation, not just the figure

For “Why did operating margins decline?”, a similarity-based retriever may rank a passage listing margin figures above the CFO’s causal explanation. In an Atlas-style workflow, a small set of question–answer examples lets PDist favor the explanatory passage because it makes the target answer easier for the reader to produce. The retriever learns to value evidence such as “driven by” rather than merely matching the question’s numbers. section 6.7

Joint training is a choice to justify, not a default requirement. In an Atlas few-shot ablation, a fixed retriever scored 42.2, PDist scored 45.7, and omitting joint pre-training scored 30.0; pre-training mattered more than fine distinctions among retriever losses. Start with a simpler pipeline when it meets the task, and evaluate joint training when answer-bearing passages repeatedly lose to lookalikes. Jointly optimizing an entire retriever–reranker–generator pipeline remains computationally expensive and an open direction. section 6.2.2 section 4.7 section 15.8.5

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Works through masked-token and reader-likelihood training signals, compares retriever losses, and leaves full-pipeline optimization as an open problem.
Examples
Uses a masked butterfly-migration sentence and a margin-decline question whose answer lies in explanatory commentary.
What it assumes
Builds on vector-based retrieval while explaining that training changes encoder weights rather than the nearest-neighbor search procedure.
What you can do after
Evaluate/understand when reader-guided retriever training warrants its cost compared with a fixed-retriever pipeline.
How it connects
Connects nearest-neighbor retrieval, masked language modeling, perplexity distillation, and retrieval-augmented generation.

Also explained in

Compare how 4 books explain end-to-end learning →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.