liveBook · Manning

DeepSeek-R1

As explained in Build a DeepSeek Model (From Scratch)

DeepSeek-R1 is a reasoning-focused large language model designed to achieve strong performance on demanding reasoning tasks.

DeepSeek-R1 is a reasoning-focused large language model built from the DeepSeek-V3 base model. Released in early 2025, it achieved results comparable to or better than leading reasoning models on benchmarks including AIME 2024 mathematics and Codeforces competitive coding. Its openly available weights and detailed reports make its training approach accessible for study. section 1.1 section 1.2.3

The full model combines Multi-Head Latent Attention with a Mixture-of-Experts architecture. Of its 671 billion total parameters, about 37 billion are activated per token. Sparse activation reduces computation per token, but the full set of weights still needs to be stored and accessible. section 8.1 section 8.1.1

R1’s staged post-training turns the base model into a more usable reasoning assistant:

  • Cold-start supervised fine-tuning uses a small set of readable, long chain-of-thought examples to establish a reasoning format. section 7.7.1
  • Reasoning-oriented reinforcement learning uses rule-based rewards and a language-consistency reward to encourage useful reasoning while reducing language mixing. section 7.7.1
  • Rejection sampling filters generated outputs for correctness and readability. Supervised fine-tuning then mixes reasoning examples with broader instruction data. section 7.7.1
  • Final reinforcement learning combines rule-based rewards for verifiable tasks with learned reward models for open-ended helpfulness and harmlessness. section 7.5.2 section 7.7.1
Figure 7.14 DeepSeek-R1 training pipeline, adapted from Guo et al., DeepSeek-R1, Figure 2. The flow from R1-Zero into R1 represents filtered training data, not direct parameter transfer.

Follow the staged pipeline and distinguish filtered training data from parameter transfer between R1-Zero and R1. section 7.7.1

R1-Zero, R1, and R1-Distill are distinct artifacts. R1-Zero tests whether reinforcement learning with verifiable rewards can produce reasoning without supervised chain-of-thought data. R1 is not simply R1-Zero trained longer: R1-Zero outputs are sampled and filtered as training data in the staged process. R1-Distill models are smaller students trained on R1-generated data. section 7.7 section 7.7.1

The deployment trade-off

At FP16 precision, the full 671-billion-parameter model’s weights alone require approximately 1,342 GB of memory; the KV cache adds further demand for long contexts. Distillation addresses this gap by using the large model as a teacher for smaller students. Released checkpoints range from 1.5B to 70B parameters and are based on Qwen2.5 and Llama3 series models. section 8.1.1 section 1.2.3

How Build a DeepSeek Model (From Scratch) covers it

How deep it goes
Explains the post-training stages, reward choices, model-family distinctions, and weight-memory costs without specifying a complete training implementation.
Examples
Cites AIME 2024, Codeforces, and 1.5B–70B distilled checkpoints as benchmark and model examples.
What you can do after
Evaluate and understand the staged training design, reward trade-offs, and deployment costs of DeepSeek-R1.
How it connects
Connects reinforcement learning, supervised fine-tuning, rejection sampling, reward models, Mixture-of-Experts, and knowledge distillation.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build a DeepSeek Model (From Scratch). The section links point to the parts of the book it draws on.