Mean reciprocal rank
Mean reciprocal rank is a ranking metric that averages, across queries, the reciprocal of the rank of each query’s first relevant result.
Machine Learning System Design
A ranking metric equal to the average reciprocal rank of the first relevant result across queries.
- How it is used
- Uses MRR as a fast offline ranking metric and potential proxy for search engagement.
- How deep it goes
- Defines reciprocal rank and discusses relevance labels and proxy limitations without a detailed derivation.
- Examples
- Uses image search results with human relevance labels, clicks, and purchases as alternative signals.
- What you can do after
- Evaluate and understand MRR as an offline proxy, then compare candidate metrics with online outcomes.
Read how Machine Learning System Design explains mean reciprocal rank →
AI Recommender Systems
Mean reciprocal rank is the average reciprocal of the position of the first relevant result across users or queries.
- How it is used
- Measures how quickly users encounter a first relevant result, especially when one good result may suffice.
- How deep it goes
- Defines the per-user score and mean, then implements first-match lookup and the no-match case at different cutoffs.
- Examples
- Uses first relevant items at positions 1 and 3, a no-result case, and a top-K metric-calculation listing.
- What you can do after
- Evaluate/understand it by calculating top-K MRR and comparing a model with baselines and other quality measures.
Read how AI Recommender Systems explains mean reciprocal rank →
The Art of AI Product Development
Mean reciprocal rank is the average reciprocal of the rank at which the first relevant result appears in a ranked result list.
- How it is used
- Evaluates how quickly search and retrieval-augmented generation systems surface a relevant document.
- How deep it goes
- Defines ranking behavior and evaluation role, but stops short of deriving the averaging formula or implementation details.
- Examples
- Uses an API integration case-study query and production signals such as clicks, query logs, dwell time, and bounce rate.
- What you can do after
- Evaluate search ranking changes and diagnose whether retrieval improvements surface relevant documents earlier.
Read how The Art of AI Product Development explains mean reciprocal rank →
Retrieval Augmented Generation, The Foundational Ideas
Mean reciprocal rank is the mean of the reciprocal ranks assigned to the first correct result across evaluation queries.
- How it is used
- Evaluates how quickly open-domain and RAG retrieval systems surface the first correct or relevant document.
- How deep it goes
- Defines reciprocal-rank averaging, gives numerical examples, and contrasts MRR with Recall@k and NDCG@k without implementing it.
- Examples
- Uses rank-one, rank-two, and rank-50 contributions plus a two-query compliance-document calculation.
- What you can do after
- Evaluate retrieval rankings with MRR and interpret its tradeoffs against Recall@k, NDCG@k, latency, and index size.
Read how Retrieval Augmented Generation, The Foundational Ideas explains mean reciprocal rank →
LLM Evaluation and Alignment, The Foundational Ideas
Mean reciprocal rank is the mean of the reciprocal ranks of the first relevant result across a set of queries.
- How it is used
- Evaluates how quickly retrieval surfaces one answer-sufficient relevant document for each query.
- How deep it goes
- Defines first-result reciprocal ranking and contrasts metric selection, but does not develop a full implementation procedure.
- Examples
- Uses a query whose first relevant document appears at rank 3, producing an MRR contribution of approximately 0.33.
- What you can do after
- Choose MRR when one relevant document suffices, and select alternative metrics for fixed-context or graded-relevance retrieval.
Read how LLM Evaluation and Alignment, The Foundational Ideas explains mean reciprocal rank →
Also covered in
- Build an Advanced RAG Application (From Scratch)In early access (MEAP)
What to know first
- precisionPrecision is the proportion of items predicted, selected, or retrieved as positive that are correct or relevant.
- recallRecall is the proportion of all relevant or actual positive items that a system successfully retrieves or correctly identifies.
- relevanceRelevance is the degree to which information, a result, or a response pertains to a query, task, purpose, or information need.
- accuracyAccuracy is the degree to which a result agrees with a specified truth, reference, or criterion, commonly measured as the proportion of evaluated predictions that are correct.
Related concepts
- rank-aware metrickind of · A rank-aware metric quantitatively evaluates the quality of an ordered list relative to desired relevance or preference while accounting for item positions.
- performancekind of · Performance is a measure of how effectively a person, process, model, or system achieves its objectives, including qualities such as output quality, speed, and resource efficiency.
- reciprocal rankuses · Reciprocal rank is a ranking metric that scores the position of the first relevant result by its reciprocal, optionally with smoothing.
- similarity searchmeasures · Similarity search is a retrieval technique that finds or ranks items by their similarity to a query, often by comparing feature or vector representations under a similarity or distance measure.
Questions about mean reciprocal rank
What is mean reciprocal rank?
Mean reciprocal rank is a ranking metric that averages, across queries, the reciprocal of the rank of each query’s first relevant result.
Which Manning books explain mean reciprocal rank?
Machine Learning System Design, AI Recommender Systems, The Art of AI Product Development, Retrieval Augmented Generation, The Foundational Ideas and LLM Evaluation and Alignment, The Foundational Ideas.
What should I know before learning mean reciprocal rank?
Manning books that teach mean reciprocal rank usually introduce precision, recall, relevance and accuracy first.
What is mean reciprocal rank a kind of?
Mean reciprocal rank is a kind of rank-aware metric. A rank-aware metric quantitatively evaluates the quality of an ordered list relative to desired relevance or preference while accounting for item positions.
What is mean reciprocal rank also called?
Mean reciprocal rank is also called MRR.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.




