liveBook · Manning

Generalization

As explained in LLM Evaluation and Alignment, The Foundational Ideas

Generalization is the ability of a model to apply learned behavior or principles to novel inputs beyond its training examples.

Generalization is applying a learned principle to novel inputs beyond the training examples. In alignment, shallow alignment matches the surface pattern of spec-aligned demonstrations but fails when the same principle appears in an unfamiliar setting. section 8.5.3

Model spec midtraining improves generalization by:

  • Inserting a midtraining stage between pretraining and alignment fine-tuning. section 8.5.3
  • Training on synthetic articles, dialogues, examples, and frequently asked questions that explain the spec, its reasons, and clause interactions. section 8.5.3
  • Testing whether behavior transfers to held-out prompts not directly covered by demonstrations. section 8.5.3
Figure 8.4 Model spec midtraining inserted between pretraining and alignment fine-tuning. The pretrained model first trains on synthetic documents about the spec (articles, dialogues, examples). The midtrained model then proceeds through the standard alignment fine-tuning steps (SFT on critique-revised demonstrations, optionally RLAIF). The midtrained model generalizes the spec to held-out prompts in ways models without MSM do not.

The pipeline places synthetic spec documents between pretraining and alignment fine-tuning, then compares generalization on held-out prompts. section 8.5.3

How LLM Evaluation and Alignment, The Foundational Ideas covers it

How deep it goes
Explains shallow failure, its cause, and a midtraining remedy, but does not detail the reported evaluation metrics.
Examples
Uses spec articles, dialogues, examples, FAQs, critique-revised demonstrations, and held-out prompts.
What it assumes
Assumes a model-spec alignment pipeline with pretraining, midtraining, and alignment fine-tuning stages.
What you can do after
Evaluate whether a model applies a specification beyond prompts resembling its demonstrations.
How it connects
Connects pretraining, alignment fine-tuning, shallow alignment, demonstrations, synthetic data, and model specifications.

Also explained in

Compare how 19 books explain generalization →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.