liveBook · Manning

Retrieval ranking

As explained in Retrieval Augmented Generation, The Foundational Ideas

Ranking is the ordering or filtering of retrieved candidates according to a relevance score, including ordering passages by relevance to a query.

A RAG system can retrieve a passage containing the answer and still fail if that passage is buried among irrelevant results. Ranking addresses this “Missed the Top Rank” failure: after retrieval fetches candidates, a scoring model filters noise and prioritizes evidence for generation. Offline indexing makes documents searchable; ranking happens in the online query pipeline. section 1.2

In a semantic retrieval stack, encoding represents queries and documents, similarity search finds candidates, and ranking orders the results by relevance. These are distinct jobs: finding potentially useful documents does not establish which should receive the generator’s attention. section 2.4.1

Rank before compressing

Suppose a question asks whether a supplier’s indemnification obligations fall under a liability cap. A query-aware compressor scores four retrieved contract articles, drops an irrelevant notices clause, and moves the liability-cap article to the front. The resulting prompt uses 212 tokens rather than 347. But compression must preserve the cap’s exception for indemnification: a displayed alternative compressed passage loses that distinction and produces the wrong answer. Inspect retained evidence and the final answer, not just the token savings. section 13.2.4 section 13.2.3

Listing 13.1 LLMLingua.
from llmlingua import PromptCompressor
from typing import List, Dict, Any
 
def compress_with_longllmlingua(
    query: str,
    documents: List[str],
    rate: float = 0.5
) -> Dict[str, Any]:
    """Compress documents with contrastive perplexity."""
 
    compressor = PromptCompressor(  #A
        model_name="Qwen/Qwen2.5-0.5B",
        use_llmlingua2=False,
        device_map="cpu"
    )
 
    compressed_output = compressor.compress_prompt(  #B
        documents,
        question=query,
        rate=rate,
        condition_compare=True,
        condition_in_question="after",
        reorder_context="sort",
        dynamic_context_compression_ratio=0.4,
        rank_method="longllmlingua"
    )
 
    return compressed_output

The LLMLingua call shows query-conditioned compression with coarse ranking, document reordering, and dynamic token budgeting configured together. section 13.2.4

A separate reranker is not the only design. RankRAG instruction-tunes one model on both passage-relevance judgments and question answering, allowing the generator to rank its own context. Its training format supports a binary judgment for one passage or identification of relevant passages in a group. This can remove a separate ranking model, but requires ranking examples alongside QA data; MS MARCO passages are given as a general-purpose source. Choose the architecture by checking answer quality as well as which component and training data it requires. section 15.3.2 section 15.3

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Explains pipeline placement, query-aware filtering and reordering, and a unified ranking-generation training format; does not derive a general relevance-scoring formula.
Examples
Uses contract-article compression and question–passage training examples to show document prioritization and relevance judgments.
What it assumes
Assumes familiarity with retrieved documents and language-model generation while explaining where ranking fits between them.
What you can do after
Apply it via LLMLingua to filter and reorder context, or evaluate RankRAG as an alternative to a separate reranker.
How it connects
Connects ranking to semantic encoding, similarity search, context compression, and answer generation.

Also explained in

Compare how 9 books explain retrieval ranking →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.