Reranking improves a retrieval pipeline when vector similarity produces a noisy order. A first-stage retriever selects top-k chunks; a cross-encoder typically scores each query–document pair jointly, then reorders those candidates. section 4.5.5
Use reranking as a quality-versus-latency decision:
- Rerank only the first-stage retriever’s top 20–50 candidates; cross-encoders add time for every pair scored. section 4.5.5
- Measure latency on the target hardware. A single pass often gives a large quality gain, but results depend heavily on the corpus. section 4.5.5
Diagnose a wrong-chunk failure
If the answer-bearing chunk is indexed but ranked below another chunk from the right document, treat ranking as the problem and consider reranking. If that chunk is absent from the index, revisit chunking instead. section 4.5.6








