Perplexity measures how predictable each token is to a language model from the tokens before it. Low perplexity marks familiar, predictable language; high perplexity marks surprising content such as numbers, named entities, or specialized terms. section 13.2
Absolute perplexity ignores the user's question. In context compression, this can remove relevant sentences because generic wording looks predictable, while retaining complex but off-topic language because it appears surprising. section 13.2.1
Use perplexity in two contrasting ways:
- Absolute perplexity scores tokens from general language statistics alone and can discard information needed for the task. section 13.2.1
- Contrastive perplexity adds the user question and keeps tokens whose predictability improves substantially under that question. section 13.2.2
- Perplexity Distillation uses the language model's signal to train a retriever to prioritize documents useful for generating the correct output. section 6.5
Question-conditioned token selection
For the question “what dose of aspirin was prescribed?”, the token “81” may have general perplexity around 5 but question-conditioned perplexity around 1, producing a strong keep signal. The connective “the” changes little and is pruned; unrelated “thrombocytopenia” remains surprising without becoming relevant and is also pruned. section 13.2.2
Browse the token-level probability form, then compare it with the query-conditioned contrastive score. section 13.2.2








