liveBook · Manning

Context recall

As explained in LLM Evaluation and Alignment, The Foundational Ideas

Context recall is a retrieval-quality metric measuring whether retrieved context contains the specific claims or information needed to produce the answer.

A retrieval-augmented generation (RAG) system can produce a faithful answer yet miss important information because retrieval did not supply it. Context recall measures whether the retrieved context contains the specific information needed for a reference answer. Unlike document recall, it focuses on answer-relevant information rather than relevant files. section 6.3 section 6.3.3

To compute context recall:

  • Prepare a ground-truth answer and split it into individual sentences. section 6.3.3
  • Check each sentence against the retrieved context. Mark it attributable if its information can be inferred from the context; otherwise, mark it not attributable. section 6.3.3
  • Divide the number of attributable sentences by the total number of ground-truth sentences. section 6.3.3

Financial research assistant

A reference answer about Q3 growth and Q4 risks has five key claims. Retrieved reports cover revenue, year-over-year growth, and the cloud services driver, but omit rising operating costs and Q4 risks. Context recall is 3/5, or 0.60: two claims lack retrieved support, regardless of how well the model uses the available context. section 6.3.3

Context recall requires a ground-truth answer, so it is typically computed offline using curated reference answers. It measures coverage of that reference, not whether the generated answer is faithful or relevant. Evaluate retrieval first: poor recall or precision can make generation results hard to interpret. section 6.3.3 section 6.4.1

Table 6.4 Interpreting metric combinations for RAG diagnosis
Faithfulness
Answer relevance
Recall
Precision
Diagnosis
High
High
High
High
Working well
Low
High
High
High
Hallucination beyond retrieved context
High
Low
High
High
Off-topic generation. Misaligned.
High
High
Low
High
Retrieval missing relevant documents
High
High
Low
Low
Retrieval is fundamentally misaligned with the query
Low
Low
Low
Low
Systemic failure

Use the diagnostic matrix to interpret recall alongside precision, faithfulness, and answer relevance. section 6.4.1

Low recall is a signal to investigate, not a complete root-cause diagnosis. For example, a keyword query may miss relevant information because its vocabulary differs from the corpus. In an agentic system, the problem may instead be query formulation; a reasoning gap that prevents a needed search may not be surfaced by context recall at all. Evaluate individual retrieval steps as well as the final output. section 6.2.1 section 6.4.1 section 6.1.3

How LLM Evaluation and Alignment, The Foundational Ideas covers it

How deep it goes
Explains sentence-level attribution and score calculation, while noting that recall alone cannot expose every agentic reasoning gap.
Examples
Calculates a 0.60 score for a financial assistant whose retrieved reports cover three of five reference claims.
What it assumes
Requires a ground-truth answer and retrieved context for reference-based scoring.
What you can do after
Apply context recall through RAGAS to identify missing reference-answer information in retrieved context.
How it connects
Connects document recall, context precision, faithfulness, answer relevance, and retrieval-augmented generation.

Also explained in

Compare how 3 books explain context recall →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.