Student model
A student model is a model trained to approximate or reproduce the outputs, capabilities, or behavior of another model that provides training guidance.
Build a Reasoning Model (From Scratch)
A smaller model trained to imitate a teacher model's reasoning traces and answers during distillation.
- How it is used
- Uses a larger reasoning teacher to transfer traces and answers into a cheaper, smaller model.
- How deep it goes
- Explains dataset construction, tokenization, cross-entropy training, validation, and hard-versus-soft distillation.
- Examples
- Uses DeepSeek-R1, Qwen3 0.6B, 12,000 MATH problems, and optional <think>...</think> reasoning tags.
- What you can do after
- Build it from scratch by preparing teacher traces, training a student, and evaluating checkpoints with validation loss and benchmarks.
Read how Build a Reasoning Model (From Scratch) explains student model →
Building Reliable AI Systems
A model trained to imitate a teacher model, often with the aim of being smaller or more efficient.
- How it is used
- Reduces production cost and latency by transferring a teacher model’s behavior to a smaller model.
- How deep it goes
- Explains prompt–response fine-tuning but stops short of detailing model internals or training implementation.
- Examples
- Uses e-commerce support questions about damaged-product returns and exchanges as training examples.
- What you can do after
- Build it from scratch by collecting teacher responses and fine-tuning a smaller model on prompt–response pairs.
Read how Building Reliable AI Systems explains student model →
Related concepts
- low-rank adaptationuses · Low-rank adaptation is a parameter-efficient fine-tuning method that adapts a model by learning low-rank updates to its weights, typically while keeping most original weights fixed.
- foundational modelcontrasts with · A base or foundation model is a pretrained model that provides learned parameters or capabilities as a starting point for reuse or adaptation to downstream tasks.
- distillationproduced by · Knowledge distillation is a training technique in which a student model learns to reproduce or approximate the outputs or behavior of a teacher model.
- teacher modeluses · A teacher model is a model whose outputs, labels, or learned behavior provide training signals or targets for another model.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.