Chapter 3: REALM: Birth of end-to-end trainable RAG
The introduction explains why retrieving similar documents is not enough to answer questions and presents REALM’s central shift toward training retrieval for usefulness to the reader.
After this chapter you will be able to
- Similarity Versus Answer Utility. Explain why a frozen, similarity-based retriever can rank a passage that resembles the question above one that contains the answer, even when the answer passage is available.
- Performance Based Retriever Training. Describe REALM’s central idea of training retrieval and reading together so the retriever favors documents that help the language model predict the correct answer. Explain that training changes the text encoder’s weights and thus the vector positions, not the nearest-neighbor search procedure.
- Two-stage ODQA pipeline. Trace how the late-2010s ODQA pipeline selected passages and extracted an answer, including the reader’s dependence on what the retriever supplied.
- Sparse and dense representations. Explain how sparse keyword representations differ from dense embeddings and why the difference matters for retrieving semantically related text.
- Asymmetric question-passage search. Distinguish symmetric text similarity from the asymmetric search required to match a short question to a longer answer passage, and explain why SBERT alone may fall short.
- DPR and the remaining gap. Describe how DPR trains separate question and passage encoders for asymmetric retrieval, and state what remained disconnected between retrieval and reading.
- Retriever and reader roles. Describe how REALM’s knowledge retriever finds documents and how its knowledge-augmented encoder uses them to predict a masked word.
- Evidence-seeking masked spans. Explain why REALM masks salient spans, such as names or dates, rather than relying on arbitrary masked words.
- Task-driven retrieval learning. Explain how masked-word prediction trains REALM’s retriever, including what the marginal objective and its top-k approximation represent.
- Pretraining and fine-tuning stages. Distinguish REALM’s unlabeled joint pretraining from its later supervised fine-tuning for a task such as open-domain question answering.
- Retrieval and answer metrics. Distinguish answer accuracy, Recall@k, and MRR, and explain why finding a correct document low in the ranking may not help the reader.
- Credible benchmark comparisons. Explain why benchmark claims need a comparable baseline, an unseen test set, and scrutiny of small gains through ablations.
- REALM question answering results. Describe REALM’s open-domain question-answering testbed and the scale of its reported improvement.
- REALM comparison lessons. Explain what the comparisons with T5-11B and ORQA suggest about external knowledge and joint training.
- Marginal log-likelihood objective. Trace how REALM combines document-retrieval and masked-word prediction probabilities, sums over candidate documents, and forms its loss.
- Unlabeled retriever training signal. Explain why masked-word prediction can train REALM’s retriever without labels identifying the correct document, using the section’s small example.
- Retrieved evidence and answer probability. Explain why REALM can predict a rare answer that a parametric-only BERT model struggles to predict, using the Fermat-prime example or the retrieved-sentence demonstration.
- External knowledge and verifiable sources. Distinguish the staleness and interpretability limits of parametric-only knowledge from the benefits of REALM’s external document repository.
- REALM++ training and evaluation gains. Explain why REALM++ improved question-answering accuracy without changing REALM’s architecture, and describe the training and evaluation changes reported here.
- REALM and RAG training contrast. Distinguish REALM’s expensive end-to-end pre-training from the later RAG approach of fine-tuning pretrained components, while noting the architecture they share.
What it builds on
- Sentence-BERT (2019): Practical sentence similarity (chapter 2) · Assumes the idea of retrieving text by semantic similarity.
- From cross-encoders to bi-encoders (chapter 2) · Assumes text can be encoded as vectors for similarity retrieval.
- The foundation for dense retrieval (chapter 2) · Introduces embeddings as a foundation for dense retrieval.
- The knowledge boundary problem (chapter 1) · The problem of knowledge bounded by a model’s training.
Section by section
Introduction
The introduction explains why retrieving similar documents is not enough to answer questions and presents REALM’s central shift toward training retrieval for usefulness to the reader.
- Similarity Versus Answer Utility: Explain why a frozen, similarity-based retriever can rank a passage that resembles the question above one that contains the answer, even when the answer passage is available.
Why it matters: Finding a plausible-looking passage is not enough if it does not help produce the answer. - Performance Based Retriever Training: Describe REALM’s central idea of training retrieval and reading together so the retriever favors documents that help the language model predict the correct answer. Explain that training changes the text encoder’s weights and thus the vector positions, not the nearest-neighbor search procedure.
Why it matters: It distinguishes learning what to retrieve from replacing the search algorithm. - External Knowledge And Verifiability: Explain how keeping knowledge in a searchable corpus permits updates without retraining the model and lets people inspect retrieved documents when checking an answer.
Why it matters: The corpus can stay current independently of the model, while its retrieved documents remain available for review.
Key terms: frozen, similarity-only retriever, FP2 (Missed the Top Rank), surface similarity, usefulness, REALM, joint pre-training, performance-based training, instrumental utility, encoder
3.1 Open-domain question answering
This section explains how open-domain question answering moved from keyword-based retrieval toward dense, question-aware retrieval, and why a gap between retriever and reader remained for REALM to address.
- Open-domain and open-book tasks: Distinguish the breadth of an open-domain task from the permission to consult external sources in an open-book task.
Why it matters: These terms describe different properties of a question-answering task and can apply together. - Two-stage ODQA pipeline: Trace how the late-2010s ODQA pipeline selected passages and extracted an answer, including the reader’s dependence on what the retriever supplied.
Why it matters: The reader’s answer quality depends on the retriever finding the right evidence first. - Sparse and dense representations: Explain how sparse keyword representations differ from dense embeddings and why the difference matters for retrieving semantically related text.
Why it matters: Representation determines whether a search can find an answer phrased differently from the question. - Asymmetric question-passage search: Distinguish symmetric text similarity from the asymmetric search required to match a short question to a longer answer passage, and explain why SBERT alone may fall short.
Why it matters: Semantic similarity alone is not the same as finding a passage that answers a differently shaped question. - DPR and the remaining gap: Describe how DPR trains separate question and passage encoders for asymmetric retrieval, and state what remained disconnected between retrieval and reading.
Why it matters: DPR addresses question-to-passage matching, while the remaining separation motivates REALM. - Role-specific embedding instructions: Apply the section’s advice to check an embedding model’s instructions and use a required query prefix on the specified side.
Why it matters: Using an embedding model without its required role instruction can silently make retrieval worse.
Key terms: open-domain, open-book, Open-Domain Question Answering (ODQA), retriever, reader, TF-IDF, BM25, BERT, sparse vectors, dense vector embeddings
3.2 Retrieval-augmented language model pre-training
This section explains how REALM jointly trains retrieval and reading through masked-word prediction, and how it makes that training practical.
- Retriever and reader roles: Describe how REALM’s knowledge retriever finds documents and how its knowledge-augmented encoder uses them to predict a masked word.
Why it matters: The two components do different jobs, but the reader’s prediction connects their training. - Evidence-seeking masked spans: Explain why REALM masks salient spans, such as names or dates, rather than relying on arbitrary masked words.
Why it matters: The choice of blank determines whether retrieval can help enough for the retriever to learn. - Task-driven retrieval learning: Explain how masked-word prediction trains REALM’s retriever, including what the marginal objective and its top-k approximation represent.
Why it matters: Retrieval learns to find information useful for the reader’s task, not merely text that looks similar. - Retriever startup and index refresh: Explain the two practical obstacles to jointly training REALM’s retriever and how REALM addresses them.
Why it matters: Joint training needs both an initially useful retriever and a workable way to keep search aligned with changing embeddings. - Pretraining and fine-tuning stages: Distinguish REALM’s unlabeled joint pretraining from its later supervised fine-tuning for a task such as open-domain question answering.
Why it matters: The two stages explain how REALM learns a general retrieval capability before adapting it to a particular application.
Key terms: knowledge retriever, dual-encoder, knowledge-augmented encoder, cross-encoder, masked language modeling, self-supervised, salient span masking, marginal log-likelihood, latent variable, top-k
3.2.1 Test results
This section explains how to read retrieval and question-answering benchmarks, then uses that lens to interpret REALM’s test results and comparisons.
- Retrieval and answer metrics: Distinguish answer accuracy, Recall@k, and MRR, and explain why finding a correct document low in the ranking may not help the reader.
Why it matters: These metrics reveal whether a system merely finds evidence or ranks it where the reader can use it. - Credible benchmark comparisons: Explain why benchmark claims need a comparable baseline, an unseen test set, and scrutiny of small gains through ablations.
Why it matters: These checks keep a benchmark gain from being mistaken for evidence of a better method. - Accuracy and efficiency trade-off: Explain why a modest retrieval gain may not justify a model’s deployment costs.
Why it matters: A better benchmark score does not automatically make a system practical. - REALM question answering results: Describe REALM’s open-domain question-answering testbed and the scale of its reported improvement.
Why it matters: The testbed and size of the gains give context to the section’s claim about REALM’s performance. - REALM comparison lessons: Explain what the comparisons with T5-11B and ORQA suggest about external knowledge and joint training.
Why it matters: The two comparisons separate the value of external knowledge access from the value of training retrieval and reading together.
Key terms: accuracy, Recall@k, MRR, reader model, baseline, dataset splits, test set, ablations, accuracy-efficiency trade-off, latency
3.3 Practical implementation with Python
This section traces REALM’s retriever scores through its marginal log-likelihood loss to show how masked-word prediction trains the retriever and reader together without document labels.
- Candidate document relevance scores: Describe how REALM’s dual-encoder scores candidate documents for a masked query and what the resulting scores represent.
Why it matters: These scores are the retriever’s contribution to the shared training objective. - Marginal log-likelihood objective: Trace how REALM combines document-retrieval and masked-word prediction probabilities, sums over candidate documents, and forms its loss.
Why it matters: The objective connects document selection to the reader’s success on the observed text. - Unlabeled retriever training signal: Explain why masked-word prediction can train REALM’s retriever without labels identifying the correct document, using the section’s small example.
Why it matters: It explains REALM’s performance-based training: a document is valuable when it helps the reader, not merely when it resembles the query.
Key terms: Knowledge Retriever, dual-encoder, retrieval_score, dot product, joint probability, marginal probability, marginal log-likelihood, gold_log_probs, reduce_logsumexp, performance-based training
3.4 REALM's enduring impact and legacy
This section shows how retrieved evidence improves REALM’s predictions, what REALM++ gained from stronger training, and why later RAG systems adopted a cheaper training approach.
- Retrieved evidence and answer probability: Explain why REALM can predict a rare answer that a parametric-only BERT model struggles to predict, using the Fermat-prime example or the retrieved-sentence demonstration.
Why it matters: It makes the benefit of grounding an answer in retrieved text observable rather than just an aggregate score. - External knowledge and verifiable sources: Distinguish the staleness and interpretability limits of parametric-only knowledge from the benefits of REALM’s external document repository.
Why it matters: External documents make answers easier to verify and knowledge easier to refresh. - REALM++ training and evaluation gains: Explain why REALM++ improved question-answering accuracy without changing REALM’s architecture, and describe the training and evaluation changes reported here.
Why it matters: It suggests improving training and evaluation before paying for an architectural redesign. - REALM and RAG training contrast: Distinguish REALM’s expensive end-to-end pre-training from the later RAG approach of fine-tuning pretrained components, while noting the architecture they share.
Why it matters: The contrast explains how retrieval augmentation became more practical without discarding REALM’s central idea.
Key terms: parametric memory, marginal likelihood, conditional probability, retriever, parametric-only models, knowledge staleness, interpretability, external document repository, modularity and updatability, REALM++
3.5 Summary
Where this is used later
- (chapter intro) (chapter 4) · Provides the REALM training approach used here as a comparison.
- Marginalization over retrieved documents (chapter 4) · Marginalization over retrieved documents builds on the idea of combining retrieval and answer probabilities.
- Case study: Open-domain question answering (chapter 4) · Interpreting retrieval and answer results in the question-answering case study.
- Ablation studies: Why each component matters (chapter 4) · Assessing what ablation studies show about system components.
- Round 2: Efficiency (chapter 4) · Interpreting efficiency alongside answer quality.
- The RAG model architecture (chapter 4) · Understanding the roles of retriever and generator in a later RAG architecture.
- Joint training without labels (chapter 6) · Providing context for joint training of retrieval and reading components.
- The retriever (chapter 4) · Provides a concrete example of a retriever assigning scores to candidate documents.
Concepts explained in this chapter
- RetrieverA retriever is a model or system component that searches an indexed collection and returns potentially relevant documents or passages.
- marginal likelihoodMarginal likelihood is the probability of an observed outcome after summing or averaging over possible latent variables or retrieved documents.
- embedding-based retrievalA bi-encoder is a neural architecture that independently encodes two inputs into representations that can be compared afterward.
- REALM++REALM++ is a method for improving REALM through changes to training scale, retrieval during evaluation, and supervision without changing its architecture.
- maximum inner product searchMaximum inner product search finds database vectors having the largest inner products with a query vector.
- knowledge-augmented encoderA knowledge-augmented encoder is an encoder that jointly processes an input and retrieved document text to predict an output.
- end-to-end learningEnd-to-end training jointly optimizes connected system components by propagating learning signals through the complete input-to-output pipeline.
- readerA reader model is a question-answering component that extracts or generates an answer from retrieved documents or passages.
- asynchronous MIPS refreshAsynchronous MIPS refresh is an index-maintenance strategy that rebuilds document embeddings and search indexes periodically while training continues on a stale index.
- masked language modelingMasked language modeling is a self-supervised training objective in which a model predicts tokens hidden from an input sequence.
- performance-based trainingA performance-based training signal is feedback derived from a system's downstream task performance and used to optimize a component.
- interpretabilityInterpretability is the degree to which a model's information sources or decision behavior can be inspected and understood.
- cross-encoderA cross-encoder jointly processes two input sequences through one encoder so that tokens from both sequences can interact directly during encoding.
- fine-tuningFine-tuning is further training of a pretrained model on task-specific or domain-specific data.
- foundation modelA foundation model is a broadly trained model that can be adapted to multiple downstream tasks.
- inverse cloze taskThe Inverse Cloze Task trains a retriever to match a sentence with the passage from which that sentence was removed.
- pretrainingPretraining is the initial training of a model on broad data before adaptation to a specialized task.
- salient span maskingSalient span masking is a masking strategy that hides information-bearing spans such as named entities or dates so retrieval affects prediction.
- similarity searchSimilarity search finds items whose representations are close to a query representation according to a similarity or distance function.
- knowledge baseThe internal knowledge base is the case study's repository of Mastercard support information supplied to the assistant.
- modularityModularity is a graph-partition quality measure comparing the observed density of within-community edges with a null-model expectation.
- parametric-only modelA parametric-only model is a model whose accessible knowledge is stored entirely in its learned parameters without consulting external information.
- stale dataStaleness is the condition in which stored or indexed information no longer reflects the current underlying source.
- BERTBERT is a bidirectional Transformer-based language model pretrained to represent text using contextual token representations.
- asymmetric searchAsymmetric search retrieves documents or passages for queries whose length, structure, and communicative role differ substantially from the targets.
- embeddingA dense vector is a vector whose coordinates are generally populated with informative numerical values.
- joint probability distributionJoint probability is the probability that multiple events or variables take specified values together.
- lexical searchLexical search retrieves text by matching words or other surface forms rather than representing meaning directly.
- marginal probabilityMarginal probability is the probability of an event obtained by summing joint probabilities over another variable.
- open-domain taskOpen-domain question answering is the task of answering questions on broad topics by retrieving evidence from a large external collection.
- latencyInference time is the time required for a trained model or system to produce an output for an input.
- Natural QuestionsNatural Questions is a dataset of naturally occurring questions paired with answers for training and evaluating question-answering systems.
- ablation studyAn ablation study is an experiment that removes or disables system components to measure their individual contributions.
- accuracyAccuracy is the proportion of evaluated system outputs judged correct according to a specified criterion.
- baselineA baseline is a reference system or result against which another system's performance is compared.
- benchmarkA benchmark is a standardized evaluation task, dataset, or reference result used to compare system performance.
- evaluation datasetAn evaluation set is a collection of examples used to measure system performance under specified evaluation conditions.
- mean reciprocal rankMean reciprocal rank is the mean of the reciprocal ranks assigned to the first correct result across evaluation queries.
- recallRecall is the proportion of relevant items that a retrieval or classification system successfully returns.
- encoderAn encoder is a model component that transforms an input sequence into a numerical representation.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
