liveBook · Manning

Reciprocal rank fusion

As explained in LLM Evaluation and Alignment, The Foundational Ideas

Reciprocal Rank Fusion combines ranked result lists by assigning each result a score based on the reciprocal of its rank in each list.

Reciprocal Rank Fusion (RRF) combines ranked result lists by giving each result a score based on the reciprocal of its rank. It commonly merges sparse and dense retrieval so precise terminology and semantic variation are both covered. section 6.5 section 6.2.1

Use RRF when retrieval methods fail in complementary ways:

  • Sparse retrieval handles precise identifiers such as product SKUs, error codes, and legal citations, but can miss paraphrases and synonyms. section 6.2.1
  • Dense retrieval handles paraphrases and conceptual queries, but can miss exact identifiers when embeddings treat them as arbitrary symbols. section 6.2.1
  • The fused ranking is evaluated with precision@k, recall@k, MAP, or nDCG; inspect duplicate results, conflicting rankings, and fusion weights as additional diagnostics. section 6.2.1

A practical retrieval choice

A query containing an exact error code may benefit from sparse retrieval, while a paraphrased conceptual query may benefit from dense retrieval. RRF combines both ranked lists instead of relying on either method alone. section 6.2.1

Table 6.2 Retrieval approaches and their failure modes
Retrieval method
Strength
Failure modes
Useful diagnostic signals
Grep / ripgrep
Exact, deterministic matching
Lexical variation; exact matches that are not task relevant
Recall across term variations; precision of exact matches
Sparse / TF-IDF / BM25
Fast, interpretable, and strong on   terminology
Paraphrases and synonyms
precision@k, recall@k, AP/MAP
Dense / vector search
Semantic similarity, handles paraphrases
Precise identifiers, exact terms
recall@k, nDCG with graded relevance, hybrid-vs-dense comparisons
Hybrid retrieval
Combines sparse and dense signals
Fusion weights, duplicate results, conflicting rankings
precision@k, recall@k, MAP, nDCG

Compare retrieval methods by strengths, failure modes, and diagnostic signals before selecting or debugging a fusion strategy. section 6.2.1

How LLM Evaluation and Alignment, The Foundational Ideas covers it

How deep it goes
Defines reciprocal-rank scoring and explains deployment rationale, but does not provide implementation pseudocode or parameter guidance.
Examples
Uses product SKUs, error codes, legal citations, paraphrases, and conceptual queries to contrast retrieval strengths.
What it assumes
Assumes familiarity with ranked retrieval lists and sparse or dense search, while contrasting their failure modes.
What you can do after
Apply it via hybrid retrieval to cover complementary sparse and dense failure modes, then diagnose results with ranking metrics.
How it connects
Connects hybrid retrieval with sparse retrieval, dense retrieval, BM25, TF-IDF, precision, recall, MAP, and nDCG.

Also explained in

Compare how 6 books explain reciprocal rank fusion →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.