liveBook · Manning

Baseline

As explained in Build a DeepSeek Model (From Scratch)

A baseline model is a reference model trained under a simpler or comparison-only procedure without teacher guidance.

A baseline model is a reference student trained on ground-truth labels without teacher guidance. Comparing it with a distilled student isolates the benefit of adding a teacher’s soft targets, especially when both students use the same architecture. section 8.6.3

Comparing student training approaches

Train the same student architecture with hard labels alone, with hard labels plus teacher soft targets at T=3, and compare both with the teacher as an upper bound. The distilled student outperforms the baseline but does not match the teacher; gains are especially pronounced on ambiguous examples and challenging benchmarks such as AIME. section 8.6.3

How Build a DeepSeek Model (From Scratch) covers it

How deep it goes
Compares training approaches and reports performance patterns without detailing the baseline training implementation.
Examples
Compares hard-label training, distillation at T=3, and teacher performance, including results on AIME.
What you can do after
Evaluate the effect of teacher guidance by comparing a student against a hard-label-only reference and the teacher.
How it connects
Connects knowledge distillation with hard labels, soft targets, and teacher models.

Also explained in

Compare how 27 books explain baseline →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build a DeepSeek Model (From Scratch). The section links point to the parts of the book it draws on.