Critic-to-generator distillation addresses a reliability gap in retrieval-augmented generation: retrieval can add noise, while generation can remain unsupported or hallucinated. It transfers a stronger critic’s evaluation behavior into a smaller generator through supervised traces. section 9.0
A practical distillation pipeline follows these steps:
- Ask a frontier critic to annotate query-answer examples with reflection tokens. section 9.0
- Fine-tune the smaller generator on those annotated traces. section 9.0
- Run the generator independently so it emits retrieval and quality-control signals during generation. section 9.0
Reflection-token transfer
A GPT-4 critic annotates a standard query-answer corpus with signals such as Retrieve, IsRel, IsSup, and IsUse. A Llama-2 7B or 13B generator learns to emit comparable decisions and critiques without calling the critic at inference time. section 9.0
