Dynamic token budgeting assigns stronger pruning to lower-ranked documents, preserving more tokens for context estimated to matter most. In a retrieval-generation pipeline, it follows coarse document ranking and precedes token pruning and context reordering. section 13.2.4
A practical compression pipeline combines four stages:
- Rank documents against the query and discard entire low-value documents. section 13.2.4
- Allocate different compression budgets by document rank, using more aggressive pruning for lower-ranked items. section 13.2.4
- Prune tokens, then reorder surviving context so important material reaches the prompt edges. section 13.2.4
Contract-question compression
Compressing four contract articles reduced 347 tokens to 212, a 1.6x ratio. The resulting text retained the indemnification and liability-cap clauses but became visibly damaged; alternate compressed outputs produced opposite answers, showing that budget selection requires task-level evaluation. section 13.2.4
Listing 13.1 LLMLingua.
from llmlingua import PromptCompressor
from typing import List, Dict, Any
def compress_with_longllmlingua(
query: str,
documents: List[str],
rate: float = 0.5
) -> Dict[str, Any]:
"""Compress documents with contrastive perplexity."""
compressor = PromptCompressor( #A
model_name="Qwen/Qwen2.5-0.5B",
use_llmlingua2=False,
device_map="cpu"
)
compressed_output = compressor.compress_prompt( #B
documents,
question=query,
rate=rate,
condition_compare=True,
condition_in_question="after",
reorder_context="sort",
dynamic_context_compression_ratio=0.4,
rank_method="longllmlingua"
)
return compressed_outputThe listing shows dynamic budgeting configured through LLMLingua's dynamic_context_compression_ratio parameter. section 13.2.4
