An instruct model is a pretrained language model trained to respond to instructions rather than merely continue text. Instruction fine-tuning (IFT), also called supervised fine-tuning (SFT), uses instruction–response examples so one model can address diverse tasks in a more reliable question-answering format. section 4.0
The adaptation keeps the next-token prediction objective but changes the training data: the model sees carefully constructed prompts paired with high-quality responses. Prompt masking can restrict learning to response tokens, while chat templates serialize system prompts, user queries, and assistant turns with special tokens. section 1.3 section 4.4
Direct answering instead of continuation
Given “What is the boiling point of water?”, a base model may continue with related questions and chemistry commentary. An instruct model answers directly: “The boiling point of water is 100C (212F) at standard atmospheric pressure.” section 4.0
Effective instruction tuning depends on:
- High-quality instruction–response data and prompt distributions that resemble downstream tasks. section 4.4
- Appropriate scale, smaller batches than pretraining, and multiturn masking choices that determine which assistant turns are trained. section 4.4
- A foundation for later preference-data collection and reinforcement learning from human feedback. section 4.4
