Instruction data pairs prompts or instructions with target responses for supervised adaptation. Applied directly on top of a base model, it teaches desired response patterns without the contrastive data required by preference fine-tuning. section 11.2.5
A practical instruction-data pipeline typically follows these steps:
- Define prompts that represent the intended training distribution, including general queries, reasoning problems, or challenging frontier tasks. section 11.2.5 section 3.2.3
- Generate or collect target completions, then filter and polish them before supervised fine-tuning. section 3.2.3 section 12.1
- Train the model on prompt–completion pairs, usually before later preference optimization or reinforcement learning. section 12.5 section 3.2.3
Constitutional critique data
Given a prompt x, repeatedly sample a principle from a constitution and ask a model to revise its latest response toward that principle. The final prompt and completion form one instruction-data example. Principles such as truthfulness or avoiding violence guide successive critiques; the resulting data can then support fine-tuning. section 12.5
Synthetic generation now dominates much instruction-data production because stronger-model distillation can provide higher-quality completions at scale than most human writers. Dataset scale ranges from Alpaca’s 52,000 prompts to more than one million samples in Tülu 3 and OpenThoughts 3, with substantially longer responses in newer datasets. section 12.1
