LLMLingua-2 compresses prompts with a BERT-style token classifier distilled from GPT-4's compression decisions. It removes less important tokens without seeing the question, producing faster, tidier text than question-conditioned compression, but potentially preserving less query-specific context. section 13.2.4
Choose between the two algorithms exposed by the same compressor interface:
- Set use_llmlingua2=True for the faster classifier-based approach; LongLLMLingua parameters are ignored. section 13.2.4
- Set use_llmlingua2=False for contrastive perplexity, which uses a causal language model and the question to score tokens. section 13.2.4
Compressing contract context
A LLMLingua-2 configuration compresses contract articles at rate=0.5. The resulting text retains limitation-of-liability terms but may become terse enough to affect interpretation, so compare its answer with the uncompressed context.
