liveBook · Manning

Teacher model

A teacher model is a model whose outputs, behavior, or learned representations provide training signals for another model.

How 6 Manning books explain it

Build a DeepSeek Model (From Scratch)

A teacher model is a model whose outputs or learned behavior provide supervision for training another model.

How it is used
Uses a capable model to supervise smaller students for more practical deployment.
How deep it goes
Develops teacher supervision from soft-target distributions through a training loop, and contrasts online logit matching with offline response generation.
Examples
Uses a toy teacher-student network and DeepSeek-R1 reasoning traces to illustrate teacher supervision.
What you can do after
Builds a toy distillation pipeline from scratch, including a frozen teacher, student, loss, and training loop.

Read how Build a DeepSeek Model (From Scratch) explains teacher model →

LLM Customization and Fine-Tuning

A model whose outputs or behavior provide targets for training a student model during distillation.

How it is used
Provides training targets for a smaller student intended to reproduce task behavior at lower serving cost.
How deep it goes
Explains teacher selection, output generation and filtering, followed by three-way evaluation and safety checks.
Examples
Uses an IT support assistant and a 50-example held-out comparison of base, teacher and student models.
What you can do after
Evaluate a teacher-student setup using behavioral, lexical and safety checks rather than assuming the student inherits every teacher capability.

Read how LLM Customization and Fine-Tuning explains teacher model →

Rearchitecting LLMs

A teacher model is a model whose outputs or behavior guide the training of a student model during knowledge distillation.

How it is used
Uses a teacher to recover pruned students or guide a specialized model replacing an agent call.
How deep it goes
Works through teacher configuration, shared-input logits, KL-divergence loss, student updates, and validation data preparation.
Examples
Uses an 18-layer Gemma teacher for a 16-layer student and a frontier model as teacher for an agent sub-task.
What you can do after
Applies knowledge distillation using PyTorch and OptiPFair to recover a pruned student from a teacher’s predictions.

Read how Rearchitecting LLMs explains teacher model →

Build a Reasoning Model (From Scratch)

A larger model whose outputs or signals are used as training targets for a smaller student model during distillation.

How it is used
Uses a stronger reasoning model to generate supervision for training a smaller, more computationally efficient student.
How deep it goes
Explains teacher outputs, reasoning traces, hard versus soft targets, and practical trade-offs, stopping short of teacher architecture internals.
Examples
Uses DeepSeek-R1 as teacher, Qwen3 0.6B as student, and MATH training problems with generated reasoning traces and answers.
What you can do after
Apply it via hard distillation by generating teacher traces and training a smaller student with supervised targets.

Read how Build a Reasoning Model (From Scratch) explains teacher model →

Retrieval Augmented Generation, The Foundational Ideas

A teacher model is a larger or more capable model whose outputs or behavior provide supervision for training a student model.

How it is used
Uses a stronger model to generate supervision for adaptive retrieval and self-critique.
How deep it goes
Explains classical and behavioral distillation, then compares critic choices and deployment tradeoffs.
Examples
Uses GPT-4, Llama 3 70B, Qwen 2.5 72B, Mixtral 8x22B, and domain-specific critics as alternatives.
What you can do after
Evaluate/understand how teacher-generated examples transfer retrieval and critique behavior to a deployable student.

Read how Retrieval Augmented Generation, The Foundational Ideas explains teacher model →

Building Reliable AI Systems

A larger, more capable model whose behavior or outputs serve as the target for a smaller model to learn.

How it is used
Supplies target outputs that a smaller student imitates to reduce deployment cost and latency.
How deep it goes
Describes prompt–response distillation and contrasts it with probability-based methods, without requiring access to logits.
Examples
Illustrates teacher outputs with e-commerce support answers, including a damaged-item return exchange.
What you can do after
Build a smaller student by collecting teacher responses to domain prompts and fine-tuning on the resulting pairs.

Read how Building Reliable AI Systems explains teacher model →

Also covered in

What to know first

Related concepts

Questions about teacher model

What is teacher model?

A teacher model is a model whose outputs, behavior, or learned representations provide training signals for another model.

Which Manning books explain teacher model?

Build a DeepSeek Model (From Scratch), LLM Customization and Fine-Tuning, Rearchitecting LLMs, Build a Reasoning Model (From Scratch), Retrieval Augmented Generation, The Foundational Ideas and Building Reliable AI Systems.

What should I know before learning teacher model?

Manning books that teach teacher model usually introduce language model, objective function, distillation and fine-tuning first.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.