liveBook · Manning

Reranking

As explained in LLM Evaluation and Alignment, The Foundational Ideas

Reranking is the process of rescoring and reordering an initial set of retrieved documents with a more selective ranking method.

Reranking rescored and reorders an initial document set with a more selective method. In RAG diagnosis, it helps recover precise identifiers missed by dense retrieval and reduce irrelevant context. section 6.4.1

Use reranking after identifying the retrieval failure:

  • Check context recall and context precision before judging generation; poor retrieval makes generation metrics unreliable. section 6.4.1
  • If dense retrieval misses precise identifiers, combine hybrid retrieval, BM25 fallback, metadata filters, entity-aware query rewriting, or reranking. section 6.4.1
  • If retrieved context contains too much irrelevant material, use reranking alongside similarity thresholds, smaller result sets, or adjusted fusion weights. section 6.4.1

Process-supervised candidate selection

Process reward model reranking compares candidate solutions on the MATH benchmark. This contrasts with outcome reward modeling and with verifiable rewards, which can avoid reward models in domains such as mathematics and code. section 7.7.6

Table 6.4 Interpreting metric combinations for RAG diagnosis
Faithfulness
Answer relevance
Recall
Precision
Diagnosis
High
High
High
High
Working well
Low
High
High
High
Hallucination beyond retrieved context
High
Low
High
High
Off-topic generation. Misaligned.
High
High
Low
High
Retrieval missing relevant documents
High
High
Low
Low
Retrieval is fundamentally misaligned with the query
Low
Low
Low
Low
Systemic failure

Compare recall, precision, faithfulness, and answer relevance to determine whether reranking addresses retrieval misalignment or whether generation remains the primary problem. section 6.4.1

How LLM Evaluation and Alignment, The Foundational Ideas covers it

How deep it goes
Treats reranking as a diagnostic remedy and candidate-selection method, but does not specify its scoring mechanics.
Examples
Uses precise-identifier retrieval failures in RAG and process reward model reranking of MATH candidate solutions.
What it assumes
Assumes familiarity with retrieval, context recall, context precision, and candidate solutions while contrasting BM25, dense retrieval, and reward models.
What you can do after
Apply reranking to improve retrieved-context precision and compare process-based candidate selection with alternative reward signals.
How it connects
Connects reranking with dense retrieval, BM25, hybrid retrieval, metadata filters, context precision, process reward models, and outcome reward models.

Also explained in

Compare how 9 books explain reranking →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.