liveBook · Manning

Mean reciprocal rank

Also called MRR

Mean reciprocal rank is a ranking metric that averages, across queries, the reciprocal of the rank of each query’s first relevant result.

How 5 Manning books explain it

Machine Learning System Design

A ranking metric equal to the average reciprocal rank of the first relevant result across queries.

How it is used
Uses MRR as a fast offline ranking metric and potential proxy for search engagement.
How deep it goes
Defines reciprocal rank and discusses relevance labels and proxy limitations without a detailed derivation.
Examples
Uses image search results with human relevance labels, clicks, and purchases as alternative signals.
What you can do after
Evaluate and understand MRR as an offline proxy, then compare candidate metrics with online outcomes.

Read how Machine Learning System Design explains mean reciprocal rank →

AI Recommender Systems

Mean reciprocal rank is the average reciprocal of the position of the first relevant result across users or queries.

How it is used
Measures how quickly users encounter a first relevant result, especially when one good result may suffice.
How deep it goes
Defines the per-user score and mean, then implements first-match lookup and the no-match case at different cutoffs.
Examples
Uses first relevant items at positions 1 and 3, a no-result case, and a top-K metric-calculation listing.
What you can do after
Evaluate/understand it by calculating top-K MRR and comparing a model with baselines and other quality measures.

Read how AI Recommender Systems explains mean reciprocal rank →

The Art of AI Product Development

Mean reciprocal rank is the average reciprocal of the rank at which the first relevant result appears in a ranked result list.

How it is used
Evaluates how quickly search and retrieval-augmented generation systems surface a relevant document.
How deep it goes
Defines ranking behavior and evaluation role, but stops short of deriving the averaging formula or implementation details.
Examples
Uses an API integration case-study query and production signals such as clicks, query logs, dwell time, and bounce rate.
What you can do after
Evaluate search ranking changes and diagnose whether retrieval improvements surface relevant documents earlier.

Read how The Art of AI Product Development explains mean reciprocal rank →

Retrieval Augmented Generation, The Foundational Ideas

Mean reciprocal rank is the mean of the reciprocal ranks assigned to the first correct result across evaluation queries.

How it is used
Evaluates how quickly open-domain and RAG retrieval systems surface the first correct or relevant document.
How deep it goes
Defines reciprocal-rank averaging, gives numerical examples, and contrasts MRR with Recall@k and NDCG@k without implementing it.
Examples
Uses rank-one, rank-two, and rank-50 contributions plus a two-query compliance-document calculation.
What you can do after
Evaluate retrieval rankings with MRR and interpret its tradeoffs against Recall@k, NDCG@k, latency, and index size.

Read how Retrieval Augmented Generation, The Foundational Ideas explains mean reciprocal rank →

LLM Evaluation and Alignment, The Foundational Ideas

Mean reciprocal rank is the mean of the reciprocal ranks of the first relevant result across a set of queries.

How it is used
Evaluates how quickly retrieval surfaces one answer-sufficient relevant document for each query.
How deep it goes
Defines first-result reciprocal ranking and contrasts metric selection, but does not develop a full implementation procedure.
Examples
Uses a query whose first relevant document appears at rank 3, producing an MRR contribution of approximately 0.33.
What you can do after
Choose MRR when one relevant document suffices, and select alternative metrics for fixed-context or graded-relevance retrieval.

Read how LLM Evaluation and Alignment, The Foundational Ideas explains mean reciprocal rank →

Also covered in

What to know first

Related concepts

Questions about mean reciprocal rank

What is mean reciprocal rank?

Mean reciprocal rank is a ranking metric that averages, across queries, the reciprocal of the rank of each query’s first relevant result.

Which Manning books explain mean reciprocal rank?

Machine Learning System Design, AI Recommender Systems, The Art of AI Product Development, Retrieval Augmented Generation, The Foundational Ideas and LLM Evaluation and Alignment, The Foundational Ideas.

What should I know before learning mean reciprocal rank?

Manning books that teach mean reciprocal rank usually introduce precision, recall, relevance and accuracy first.

What is mean reciprocal rank a kind of?

Mean reciprocal rank is a kind of rank-aware metric. A rank-aware metric quantitatively evaluates the quality of an ordered list relative to desired relevance or preference while accounting for item positions.

What is mean reciprocal rank also called?

Mean reciprocal rank is also called MRR.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.