Fine-tuning further trains a pretrained foundation model on a smaller, labeled dataset tailored to a task or domain. It follows broad pretraining and adapts the model for downstream performance, such as instruction following or text classification. section 1.3 section 1.8
A practical fine-tuning workflow is:
- Start with a pretrained LLM rather than repeating the expensive large-corpus pretraining stage. section 1.5
- Prepare a smaller labeled dataset representing the target task, such as instructions and expected responses. section 1.8 section 7.5
- Load the pretrained model and reuse much of the preprocessing and training code used for pretraining. section 7.5
- Compare the model's output before and after training to establish a baseline and assess the effect of fine-tuning. section 7.5
Instruction-following adaptation
A pretrained GPT model receives the instruction, "Convert the active sentence to passive: 'The chef cooks the meal every day.'" Fine-tuning on instruction-response pairs aims to make the model produce an appropriate answer instead of continuing the prompt as ordinary text. section 1.7 section 7.5
Figure 7.15 The three-stage process for instruction fine-tuning an LLM. After the dataset preparation, the process of fine-tuning an LLM for instruction-following begins with loading a pretrained LLM, which serves as the foundation for subsequent training.
Use the three-stage workflow figure to locate dataset preparation, pretrained-model loading, and subsequent instruction fine-tuning. section 7.5
