Teacher model
A teacher model is a model whose outputs, labels, or learned behavior provide training signals or targets for another model.
Build a Reasoning Model (From Scratch)
A larger model whose outputs or signals are used as training targets for a smaller student model during distillation.
- How it is used
- Uses a stronger reasoning model to generate supervision for training a smaller, more computationally efficient student.
- How deep it goes
- Explains teacher outputs, reasoning traces, hard versus soft targets, and practical trade-offs, stopping short of teacher architecture internals.
- Examples
- Uses DeepSeek-R1 as teacher, Qwen3 0.6B as student, and MATH training problems with generated reasoning traces and answers.
- What you can do after
- Apply it via hard distillation by generating teacher traces and training a smaller student with supervised targets.
Read how Build a Reasoning Model (From Scratch) explains teacher model →
Building Reliable AI Systems
A larger, more capable model whose behavior or outputs serve as the target for a smaller model to learn.
- How it is used
- Supplies target outputs that a smaller student imitates to reduce deployment cost and latency.
- How deep it goes
- Describes prompt–response distillation and contrasts it with probability-based methods, without requiring access to logits.
- Examples
- Illustrates teacher outputs with e-commerce support answers, including a damaged-item return exchange.
- What you can do after
- Build a smaller student by collecting teacher responses to domain prompts and fine-tuning on the resulting pairs.
Read how Building Reliable AI Systems explains teacher model →
Related concepts
- JSON arrayproduces · A JSON array is an ordered JSON value containing zero or more JSON values, represented as a comma-separated sequence enclosed in square brackets.
- distillationused by · Knowledge distillation is a training technique in which a student model learns to reproduce or approximate the outputs or behavior of a teacher model.
- student modelused by · A student model is a model trained to approximate or reproduce the outputs, capabilities, or behavior of another model that provides training guidance.
- no_grad()used by · no_grad() is a PyTorch context manager and decorator that disables autograd gradient tracking for enclosed tensor operations.
- synthetic datauses · Synthetic data is data generated artificially rather than collected directly from the target process, often to approximate its properties or meet specified requirements.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.