Mean reciprocal rank
Mean reciprocal rank is a ranking metric that averages, across queries, the reciprocal of the rank of each query’s first relevant result.
Machine Learning System Design
A ranking metric equal to the average reciprocal rank of the first relevant result across queries.
- How it is used
- Evaluates how early a search results page presents its first relevant photo and serves as a candidate offline proxy for live outcomes.
- How deep it goes
- Defines the reciprocal-rank average and distinguishes relevance signals, but does not develop a detailed calculation procedure.
- Examples
- Uses human-labeled, click-based, and purchase-based MRR variants for photo search, alongside results from an existing non-ML search algorithm.
- What you can do after
- Evaluate/understand search rankings with MRR while checking whether the chosen relevance signal tracks the intended live metric.
Read how Machine Learning System Design explains mean reciprocal rank →
The Art of AI Product Development
Mean reciprocal rank is the average reciprocal of the rank at which the first relevant result appears in a ranked result list.
- How it is used
- Evaluates how quickly search and retrieval-augmented generation systems surface a relevant document.
- How deep it goes
- Defines ranking behavior and evaluation role, but stops short of deriving the averaging formula or implementation details.
- Examples
- Uses an API integration case-study query and production signals such as clicks, query logs, dwell time, and bounce rate.
- What you can do after
- Evaluate search ranking changes and diagnose whether retrieval improvements surface relevant documents earlier.
Read how The Art of AI Product Development explains mean reciprocal rank →
Related concepts
- performancekind of · Performance is the measurable effectiveness or operational behavior of a system under specified conditions, assessed through relevant qualities such as task results, speed, responsiveness, throughput, or resource use.
- similarity searchmeasures · Similarity search is a retrieval technique that finds or ranks items by comparing their representations with a query according to a similarity or distance measure.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.