Distillation trains a smaller student model to mimic labels or signals produced by a more capable teacher, or oracle. The oracle runs offline on training data; the student then supplies the signal in deployment, where the oracle may be too slow, costly, or unavailable. In an RL system, this keeps the language model in the representation role: it produces state information, while the RL agent makes decisions. section 12.3 section 12.10
A practical distillation workflow is to:
- Use the oracle to label curated training data offline. section 12.11 section 12.3
- Fine-tune a smaller model to reproduce those labels, then deploy the student instead of the oracle. section 12.3
- Measure how much of the oracle’s signal survives and whether it provides decision value. section 12.3
News signals for a portfolio manager
A frontier model such as DeepSeek reads news and generates reference market signals. It labels data offline; a smaller language model learns to reproduce those signals and supplies them to the portfolio manager’s state in deployment. The experiment measures whether useful signal survives the transfer. section 12.3
Figure 12.4 The distillation framing: an expensive oracle, a cheap deployable student.

The figure contrasts an expensive oracle that labels data offline with a cheaper student used in deployment. section 12.3













