liveBook · Manning

Reranking

As explained in Building Reliable AI Systems

Reranking is the reordering of retrieved results using a further relevance-scoring step.

Reranking reorders retrieved candidates to improve precision when relevant documents are present but buried below less relevant results. In multistage retrieval, it follows broad retrieval and progressive filtering. section 4.3.3

A reranking model can score candidates in a different way from the initial retriever:

  • Bi-encoders separately encode queries and documents, then compare their vectors; precomputed document embeddings make this efficient, but the query and document do not interact directly. section 4.3.3
  • Cross-encoders process a query and document together and return a relevance score, capturing term interactions such as “work accommodation” matching “ADA.” section 4.3.3

Remote-work policy search

A hybrid retriever finds documents about remote work for employees with disabilities, but the highly relevant ADA accommodation result ranks third. Reranking aims to place it higher. section 4.3.3

1. "Company holiday and leave policies for 2023" [marginally relevant]
2. "Benefits enrollment deadlines" [not relevant]
3. "Employees may request remote work accommodations under ADA" [highly
    relevant!]
4. "General remote work guidelines" [somewhat relevant]
5. "New office opening announcement" [not relevant]

The ranked results show the highly relevant ADA accommodation document below less-relevant matches. section 4.3.3

How Building Reliable AI Systems covers it

How deep it goes
Contrasts bi-encoder and cross-encoder scoring and explains query-document interaction, without providing implementation code.
Examples
Uses a remote-work accommodation query where the relevant ADA document is initially ranked third.
What you can do after
Evaluate whether a ranking stage promotes highly relevant documents above marginal matches.
How it connects
Connects reranking to hybrid retrieval, multistage retrieval, embeddings, bi-encoders, and cross-encoders.

Also explained in

Compare how 3 books explain reranking →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Building Reliable AI Systems. The section links point to the parts of the book it draws on.