Generalization is applying a learned principle to novel inputs beyond the training examples. In alignment, shallow alignment matches the surface pattern of spec-aligned demonstrations but fails when the same principle appears in an unfamiliar setting. section 8.5.3
Model spec midtraining improves generalization by:
- Inserting a midtraining stage between pretraining and alignment fine-tuning. section 8.5.3
- Training on synthetic articles, dialogues, examples, and frequently asked questions that explain the spec, its reasons, and clause interactions. section 8.5.3
- Testing whether behavior transfers to held-out prompts not directly covered by demonstrations. section 8.5.3
Figure 8.4 Model spec midtraining inserted between pretraining and alignment fine-tuning. The pretrained model first trains on synthetic documents about the spec (articles, dialogues, examples). The midtrained model then proceeds through the standard alignment fine-tuning steps (SFT on critique-revised demonstrations, optionally RLAIF). The midtrained model generalizes the spec to held-out prompts in ways models without MSM do not.
The pipeline places synthetic spec documents between pretraining and alignment fine-tuning, then compares generalization on held-out prompts. section 8.5.3


















