Coarse document ranking is the first filtering stage in a prompt-compression pipeline. It scores whole documents against the query and removes low-ranked documents before dynamic budget allocation, token pruning, and reordering. section 13.2.4
The ranking step sits between retrieval and generation, reducing prompt size before finer-grained compression. Its production value is latency and context savings, but removing a document can change the answer when ranking or later compression loses relevant evidence. section 13.2.4
Contract-question compression
Given four contract articles and the question of whether supplier indemnification falls under a liability cap, the coarse ranker drops the notices clause entirely. The remaining material compresses from 347 to 212 tokens, or 1.6x, while retaining the relevant liability and indemnification clauses. section 13.2.4
Listing 13.1 LLMLingua.
from llmlingua import PromptCompressor
from typing import List, Dict, Any
def compress_with_longllmlingua(
query: str,
documents: List[str],
rate: float = 0.5
) -> Dict[str, Any]:
"""Compress documents with contrastive perplexity."""
compressor = PromptCompressor( #A
model_name="Qwen/Qwen2.5-0.5B",
use_llmlingua2=False,
device_map="cpu"
)
compressed_output = compressor.compress_prompt( #B
documents,
question=query,
rate=rate,
condition_compare=True,
condition_in_question="after",
reorder_context="sort",
dynamic_context_compression_ratio=0.4,
rank_method="longllmlingua"
)
return compressed_outputShows coarse ranking as the document-level stage of an LLMLingua compression call, alongside dynamic budgeting, token pruning, and context reordering. section 13.2.4
