A causal language model predicts next-token probabilities from preceding tokens. In prompt compression, it acts as a proxy scorer: comparing probabilities with and without the question helps identify context worth preserving. section 13.2.4
Use a causal language model when:
- Contrastive perplexity must judge document tokens in relation to a query. section 13.2.4
- A small proxy model can run before generation to reduce context length and token cost. section 13.2.4
- The additional scoring latency is justified by documents that exceed the generator's effective context window. section 13.2.4
Proxy scorer in LLMLingua
Listing 13.1 configures Qwen/Qwen2.5-0.5B as the causal proxy on a laptop CPU, then compresses documents with the question attached. The alternative LLMLingua-2 path uses a BERT-style token classifier and does not see the question. section 13.2.4




