liveBook · Manning

Reader

As explained in Retrieval Augmented Generation, The Foundational Ideas

A reader model is a question-answering component that extracts or generates an answer from retrieved documents or passages.

A reader is the question-answering component that receives retrieved passages and extracts or generates an answer. In a two-stage pipeline, retrieval decides what the reader can see. A capable reader cannot answer from a passage the retriever missed. section 3.1

When the right passage never arrives

An employee asks how much they can put into a 401(k). The relevant policy says “maximum employee deferral,” not “put into.” A keyword retriever may miss it; a BERT reader that could recognize the answer never gets the chance. section 3.1

Readers need not produce answers the same way. A fine-tuned BERT reader identifies an answer span in text. Atlas instead uses a T5-v1.1 sequence-to-sequence reader to generate an answer from retrieved documents. Its Fusion-in-Decoder structure supports processing multiple documents without simply concatenating them into one long input. section 3.1 section 6.1 section 6.5

1. L2=0.331  AWS operating margin was 25.1% in Q3 2024, down from 28.
2. L2=0.937  The decline in operating margins was primarily driven by
3. L2=1.300  Net sales grew 11% year-over-year to $158.9B in Q3 2024.

The ranking shows a passage with matching figures above the passage explaining the decline. The explanation exists, but at retrieval depth one the reader sees only the less useful result. section 6.7

More retrieved passages do not automatically help. With only answer-bearing passages, even noise-sensitive readers can perform well; irrelevant passages are the harder test. Better retrieval may raise recall while adding noise that offsets the reader’s gains. Reranking and context compression can help before increasing retrieval depth. section 15.1.3 section 15.5.3

Choose and improve a reader according to the failure you observe:

  • For a new system, measure answer performance across retrieval depths from k=1 to k=20. A peak followed by decline signals noise sensitivity; continued improvement followed by a plateau supports retrieving more. section 15.5.3
  • When distractor passages are the problem, RAFT fine-tunes on answer-bearing passages mixed with distractors and answers that quote the relevant text. section 15.3.3
  • With few labeled examples, consider a jointly pre-trained retriever and reader. Perplexity Distillation uses the reader’s likelihood of the correct output to teach the retriever which documents are useful. section 6.5 section 6.7

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Contrasts span extraction with multi-document generation, then develops noise-robustness and few-shot training choices rather than a from-scratch reader implementation.
Examples
Uses a 401(k) vocabulary mismatch and a financial-report ranking failure to show how answer-bearing evidence can miss the reader.
What you can do after
Diagnose/debug with retrieval-depth profiling, then choose reranking, context compression, or RAFT when irrelevant passages impair answers.
How it connects
Links readers to sparse retrieval, dense retrieval, Fusion-in-Decoder, reranking, context compression, and retrieval-augmented fine-tuning.

Also explained in

Compare how 3 books explain reader →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.