Distillation compresses a large language model into a smaller student model by training it to reproduce the teacher’s outputs and behavior. The student targets faster, less resource-intensive inference while preserving as much accuracy, reasoning ability, and alignment as possible. Because the student can learn without the teacher’s original training data, distillation supports deployment on commodity GPUs and edge devices. section 9.1 section 9.1.2
A typical distillation setup separates the roles and learning signals:
- Use a large pretrained model as the teacher and a smaller, more efficient model as the student. section 9.1.2
- Choose the smaller model when lower memory use, cost, or latency matters more than the teacher’s full scale. section 9.1.2
Distilled model families
Model families apply the pattern at different deployment scales: Meta’s Llama 4 Behemoth is associated with smaller Scout and Maverick models; GPT-5 with GPT-5 mini; Claude Opus with Claude Sonnet; Gemini 1.5 Pro with Gemini 1.5 Flash; and Gemma 3 4B with Gemma 3 270M for on-device and edge use.
