liveBook · Manning

Cold-start dataset

As explained in Build a DeepSeek Model (From Scratch)

A cold-start dataset is a relatively small initial dataset of curated examples used to establish a model's behavior before later optimization.

A cold-start dataset is a small, curated set of supervised examples that establishes a model’s initial behavior before reinforcement-learning optimization. In DeepSeek-R1, the examples teach a readable reasoning format rather than transferring model parameters. section 7.7.1 section 1.2.3

The dataset addresses several practical weaknesses of starting reinforcement learning from an unstructured model:

  • It establishes response style, formatting, and a structured solution with a final-answer region. section 7.7.2
  • It improves readability and reduces language mixing observed in purely reinforcement-trained behavior. section 7.7.1 section 7.7.2
  • It reduces wasted exploration by making outputs easier for verifiers to parse, producing a cleaner reinforcement-learning signal. section 7.7.2
  • It complements reinforcement learning: supervised fine-tuning imitates known formats, while reinforcement learning optimizes outcomes that can be scored. section 7.7.2

Reasoning-format initialization

A small collection of long, readable chain-of-thought examples can teach the model to produce a structured solution followed by a final answer before reasoning-oriented reinforcement learning begins. section 7.7.1 section 7.7.2

Figure 7.14 DeepSeek-R1 training pipeline, adapted from Guo et al., DeepSeek-R1, Figure 2. The flow from R1-Zero into R1 represents filtered training data, not direct parameter transfer.

The pipeline distinguishes filtered training-data flow from direct parameter transfer and places cold-start supervision before reasoning reinforcement learning. section 7.7.1

How Build a DeepSeek Model (From Scratch) covers it

How deep it goes
Explains purpose, benefits, and pipeline position, but does not specify dataset construction procedures or example counts.
Examples
Uses curated long chain-of-thought examples and structured solutions containing a final-answer region.
What it assumes
Assumes reinforcement learning and supervised fine-tuning, while contrasting their roles in post-training.
What you can do after
Apply supervised initialization to make later reinforcement-learning rollouts more readable, structured, and easier to verify.
How it connects
Connects supervised fine-tuning with reinforcement learning, rejection sampling, verifiers, and reward models.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build a DeepSeek Model (From Scratch). The section links point to the parts of the book it draws on.