liveBook · Manning

REALM++

As explained in Retrieval Augmented Generation, The Foundational Ideas

REALM++ is a method for improving REALM through changes to training scale, retrieval during evaluation, and supervision without changing its architecture.

A jointly trained retriever and reader can still underperform if fine-tuning leaves their capacity unused. REALM++ improves REALM without changing its architecture. It addresses undertraining and reports an accuracy gain of roughly 5.5 percentage points on a question-answering task. section 3.4

The changes focus on training and evaluation rather than a new retriever or reader:

  • Increase the effective fine-tuning batch size. More diverse question–document pairings and hard negatives in each update help the retriever distinguish useful passages. section 3.4
  • Use exact maximum inner product search (MIPS), instead of approximate nearest-neighbor search, for final evaluation. This checks performance when approximate retrieval does not affect which documents are selected. section 3.4
  • Strengthen supervision alongside the scaling changes. The reported gain is for the improved approach as a whole, not an isolated estimate for this lever. section 3.4

The practical lesson is to test whether fine-tuning and retrieval evaluation are limiting an existing system before replacing its architecture. Exact MIPS is described for final evaluation; that choice alone does not establish a production retrieval strategy. section 3.4

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Identifies fine-tuning scale, exact MIPS evaluation, and stronger training signals, but does not specify the supervision mechanism.
Examples
Uses a roughly 5.5-percentage-point question-answering accuracy gain to illustrate the effect of improving training and evaluation.
What it assumes
Assumes familiarity with REALM’s jointly trained retriever and reader and introduces the improvement levers in that context.
What you can do after
Evaluate/understand whether fine-tuning scale and final retrieval accuracy merit attention before an architecture change.
How it connects
Connects joint retriever–reader training with effective batch size, hard negatives, approximate nearest-neighbor search, and maximum inner product search.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.