liveBook · Manning

Demonstration

As explained in LLM Evaluation and Alignment, The Foundational Ideas

A demonstration is a training example showing a model how to respond in a particular situation or case.

A demonstration is a training example showing a model how to respond in a particular situation or case. Alignment fine-tuning uses demonstrations, including critique-revised examples, to teach desired behavior. section 8.5.3

Demonstrations can create shallow alignment when a model learns their surface patterns instead of the underlying principle. It may respond correctly to familiar prompts but generalize poorly to new settings. section 8.5.3

Use demonstrations diagnostically by checking:

  • Whether the response matches a familiar training pattern or applies the intended principle to a held-out prompt. section 8.5.3
  • Whether additional material explains the specification and its rationale before critique-revised demonstrations are used. section 8.5.3
Figure 8.4 Model spec midtraining inserted between pretraining and alignment fine-tuning. The pretrained model first trains on synthetic documents about the spec (articles, dialogues, examples). The midtrained model then proceeds through the standard alignment fine-tuning steps (SFT on critique-revised demonstrations, optionally RLAIF). The midtrained model generalizes the spec to held-out prompts in ways models without MSM do not.

Shows demonstrations positioned in the alignment pipeline after model-spec midtraining and before optional RLAIF. section 8.5.3

How LLM Evaluation and Alignment, The Foundational Ideas covers it

How deep it goes
Defines demonstrations and diagnoses their limits, but stops short of specifying how to author or curate them.
Examples
Uses critique-revised demonstrations and evaluates generalization on prompts not directly covered by demonstration data.
What you can do after
Evaluate whether demonstrations teach underlying principles rather than merely reproducing familiar response patterns.
How it connects
Connects demonstrations with alignment fine-tuning, shallow alignment, model specifications, supervised fine-tuning, and RLAIF.

Also explained in

Compare how 9 books explain demonstration →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.