liveBook · Manning

Contrastive perplexity

As explained in Retrieval Augmented Generation, The Foundational Ideas

Contrastive perplexity is a query-conditioned relevance measure based on the reduction in a token's perplexity after the user question is added to its conditioning context.

A retrieved passage can contain the answer yet lose it during compression. Pruning by absolute perplexity favors unusual tokens and can discard ordinary-looking answer text while retaining surprising but irrelevant words. Contrastive perplexity instead asks whether the user’s question makes each token more predictable. section 13.2.1 section 13.2.2

s(xᵢ) = PPL(xᵢ | x<ᵢ) − PPL(xᵢ | x<ᵢ, q)

Score a token by subtracting its perplexity when conditioned on the question from its perplexity given only preceding tokens. A large positive difference is a keep signal; a score near zero suggests the question adds little reason to retain it. section 13.2.2

Finding the dose, not the unusual word

For “what dose of aspirin was prescribed?”, the token “81” in “81 mg of aspirin daily” might drop from perplexity 5 to 1 when the question is supplied, giving a score near 4. A common “the” changes little and scores near zero. An off-topic medical term can have high absolute perplexity but still score low if the question does not make it more predictable. section 13.2.2

In LongLLMLingua, this token score is one stage of a larger process. Document-level filtering first removes irrelevant passages; reordering places strong passages at prompt boundaries; iterative pruning then applies contrastive perplexity within 200-token segments. A final recovery pass aligns damaged entity fragments in the generated answer with the uncompressed source. section 13.2.3

Listing 13.1 LLMLingua.
from llmlingua import PromptCompressor
from typing import List, Dict, Any
 
def compress_with_longllmlingua(
    query: str,
    documents: List[str],
    rate: float = 0.5
) -> Dict[str, Any]:
    """Compress documents with contrastive perplexity."""
 
    compressor = PromptCompressor(  #A
        model_name="Qwen/Qwen2.5-0.5B",
        use_llmlingua2=False,
        device_map="cpu"
    )
 
    compressed_output = compressor.compress_prompt(  #B
        documents,
        question=query,
        rate=rate,
        condition_compare=True,
        condition_in_question="after",
        reorder_context="sort",
        dynamic_context_compression_ratio=0.4,
        rank_method="longllmlingua"
    )
 
    return compressed_output

The Python example calls llmlingua’s PromptCompressor with a question, documents, and a compression rate. Its use_llmlingua2=False setting selects the question-conditioned LongLLMLingua pipeline; the alternative LLMLingua-2 classifier does not see the question. section 13.2.4

Use the extra proxy-model scoring when retrieved context is long, costly, or vulnerable to lost-in-the-middle errors—not merely because compression is available. The proxy pass adds latency for short, well-targeted documents. Inspect retained answer evidence as well as token savings: aggressive compression can remove meaning. section 13.2.4 section 13.9

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Defines a perplexity difference, works through token scores, and places scoring inside a compression pipeline without deriving proxy-model internals.
Examples
Uses an aspirin-dose question to compare relevant, common, and off-topic tokens; tests compressed contract language against a liability-cap question.
What it assumes
Introduces ordinary conditional perplexity before the contrastive score and requires a causal language model for next-token scoring.
What you can do after
Apply it via llmlingua’s PromptCompressor to prune documents for a supplied question, then check token savings and retained answer evidence.
How it connects
Connects conditional perplexity to context compression, document filtering, prompt reordering, and extractive pruning.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.