Reinforcement learning trains a policy from reward signals produced by interaction with an environment. In retrieval-augmented systems, it replaces per-document likelihood or human compression labels with task-level signals such as answer quality. section 6.6.2 section 13.4.1
CORE illustrates the workflow for a compressor: distill high-quality summaries first, then refine the policy with the downstream language model's exact-match and F1 scores. Group Relative Policy Optimization samples candidate summaries, compares their normalized group rewards, and avoids a separate critic model. section 13.4.1 section 13.4.1
The approach appears in several retrieval and compression decisions:
- Train a search policy against the final answer rather than optimizing per-document likelihood. section 6.6.2
- Train a compressor to optimize downstream task performance, reaching a three percent compression ratio while improving exact-match scores over an uncompressed baseline. section 13.4.1
- Use reinforcement learning when generic compression quality or token pruning misses the facts required for multi-hop synthesis. section 13.4.1



























