liveBook · Manning

Coarse document ranking

As explained in Retrieval Augmented Generation, The Foundational Ideas

Coarse document ranking orders whole documents by estimated relevance before finer-grained compression.

Coarse document ranking is the first filtering stage in a prompt-compression pipeline. It scores whole documents against the query and removes low-ranked documents before dynamic budget allocation, token pruning, and reordering. section 13.2.4

The ranking step sits between retrieval and generation, reducing prompt size before finer-grained compression. Its production value is latency and context savings, but removing a document can change the answer when ranking or later compression loses relevant evidence. section 13.2.4

Contract-question compression

Given four contract articles and the question of whether supplier indemnification falls under a liability cap, the coarse ranker drops the notices clause entirely. The remaining material compresses from 347 to 212 tokens, or 1.6x, while retaining the relevant liability and indemnification clauses. section 13.2.4

Listing 13.1 LLMLingua.
from llmlingua import PromptCompressor
from typing import List, Dict, Any
 
def compress_with_longllmlingua(
    query: str,
    documents: List[str],
    rate: float = 0.5
) -> Dict[str, Any]:
    """Compress documents with contrastive perplexity."""
 
    compressor = PromptCompressor(  #A
        model_name="Qwen/Qwen2.5-0.5B",
        use_llmlingua2=False,
        device_map="cpu"
    )
 
    compressed_output = compressor.compress_prompt(  #B
        documents,
        question=query,
        rate=rate,
        condition_compare=True,
        condition_in_question="after",
        reorder_context="sort",
        dynamic_context_compression_ratio=0.4,
        rank_method="longllmlingua"
    )
 
    return compressed_output

Shows coarse ranking as the document-level stage of an LLMLingua compression call, alongside dynamic budgeting, token pruning, and context reordering. section 13.2.4

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Explains document-level dropping within a four-stage pipeline but does not detail the ranker's scoring algorithm.
Examples
Uses four contract articles and removes the notices clause before compressing the remaining context.
What it assumes
Assumes a query and retrieved documents, while placing the compressor between retrieval and generation.
What you can do after
Apply it via LLMLingua to filter retrieved documents before finer-grained prompt compression.
How it connects
Connects retrieval, generation, dynamic budget allocation, token pruning, reordering, and contrastive perplexity.

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.