A smaller retrieved context can make a generator faster and cheaper to run—but it can also remove the words needed for a correct answer. Compression ratio measures the size reduction: divide the original token count by the compressed token count. A 4x ratio means the compressed context is one-quarter as long. The ratio says nothing by itself about whether the answer survived. section 13.2.4 section 13.7
More compression, wrong answer
In a contract question, LongLLMLingua reduced 347 tokens to 212, about 1.6x. LLMLingua-2 reduced 334 to 109, about 3.1x, but dropped “Except with respect to.” The surviving clause appeared to put indemnification obligations under a liability cap when it actually exempted them. The answer was fluent and cited, yet reversed. Protecting digits and article markers restored citations, not the missing exception. section 13.2.4
Different compressors reach different ratios by different routes. LongLLMLingua scores content against the question; AttentionRAG keeps high-scoring sentences using attention from an open-weight pruning model. Trained synthesis can compress further by rewriting retrieved content. Check how each result expresses size: CORE reports a three-percent compression setting, while BRIEF-Pro reports a 32x ratio; those figures should not be read as the same notation. section 13.2.4 section 13.3 section 13.4.1
Choose a ratio against the task, not as a standalone target:
- Start around 4x. Compare compressed and uncompressed answers on held-out queries with known answers; increase the ratio only while the task metric holds. section 13.7
- Force-keep terms whose loss could change meaning, and inspect compressed passages—not just token counts or fluent answers. section 13.7 section 13.2.4
- Account for method limits: AttentionRAG's ratio varies by document and cannot be set directly, complicating hard token budgets. section 13.3
Listing 13.2 Analyzing the result.
def analyze_compression_results(output: Dict[str, Any]) -> None:
"""Print the token reduction and content changes."""
original_count = output.get("origin_tokens", 0)
compressed_count = output.get("compressed_tokens", 0)
actual_ratio = output.get("ratio", "0x")
print(f"Original token count: {original_count}")
print(f"Compressed token count: {compressed_count}")
print(f"Effective compression ratio: {actual_ratio}")
final_prompt = output.get("compressed_prompt", "") #A
print(f"\nSnippet: {final_prompt[:200]}...")The result-inspection code prints original tokens, compressed tokens, the effective ratio, and a prompt snippet so size reduction can be checked alongside retained content. section 13.2.4

