liveBook · Manning

Reranking

As explained in LLM Customization and Fine-Tuning

A retrieval technique that reorders an initial set of results using a further relevance assessment.

Reranking improves a retrieval pipeline when vector similarity produces a noisy order. A first-stage retriever selects top-k chunks; a cross-encoder typically scores each query–document pair jointly, then reorders those candidates. section 4.5.5

Use reranking as a quality-versus-latency decision:

  • Rerank only the first-stage retriever’s top 20–50 candidates; cross-encoders add time for every pair scored. section 4.5.5
  • Measure latency on the target hardware. A single pass often gives a large quality gain, but results depend heavily on the corpus. section 4.5.5

Diagnose a wrong-chunk failure

If the answer-bearing chunk is indexed but ranked below another chunk from the right document, treat ranking as the problem and consider reranking. If that chunk is absent from the index, revisit chunking instead. section 4.5.6

How LLM Customization and Fine-Tuning covers it

How deep it goes
Defines joint query-document scoring and practical candidate and latency limits, but gives no model-specific implementation.
Examples
Uses the wrong-chunk failure—an answer-bearing chunk is indexed but ranked too low—to distinguish ranking problems from chunking problems.
What it assumes
Assumes a first-stage retriever already returns candidate chunks for further relevance scoring.
What you can do after
Apply reranking by rescoring only the first-stage retriever’s top 20–50 candidates and measuring latency on target hardware.
How it connects
Connects vector search, cross-encoders, and chunking within retrieval-augmented generation.

Also explained in

Compare how 9 books explain reranking →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Customization and Fine-Tuning. The section links point to the parts of the book it draws on.