Suppose a general-purpose language model handles everyday questions but struggles with your domain. Fine-tuning continues training a pretrained model on task-specific data. It reuses what the model learned from a large corpus, so adapting it for work such as classification or question answering generally takes less data, time, and compute than training from scratch. section 5.2.2 section 2.3.3
A practical fine-tuning workflow starts before the training loop:
- Choose a pretrained model that fits the task and available resources; decide what behavior you want to improve. section 5.2.2
- Acquire, clean, and curate representative training data. Prepare a tokenizer and evaluation data rather than treating either as an afterthought. section 4.0 section 5.5
- Continue training on the curated data, then evaluate whether the adapted model meets the task’s needs. Fine-tuning can use the same basic training loop as training from scratch, with a fraction of the data requirement. section 5.2.2 section 4.6
Figure 5.1 The training life cycle of an LLM. We start by creating a foundation model based on a large corpus of text, which we later finetune using a curated dataset for a specific task. We can then further improve the model by using the model itself and techniques like prompting to enhance or enlarge our curated dataset.
The training lifecycle places fine-tuning between foundation-model training on a large corpus and prompting for a particular response. section 5.2
Data shape matters as much as data volume. An instruction-tuning record can pair an instruction and optional input with the response you want. For a translation task, the instruction requests a translation, the input holds the sentence, and the response supplies its translation. Curate examples that reflect the speech acts and situations the model will actually encounter; poor distribution can teach the wrong behavior. section 4.3.2 section 4.6
Two ways to adapt an existing model
A dataset of Slack messages can be used to train a conversational persona. For a hosted route, the OpenAI Python client uploads a JSONL training file, creates a fine-tuning job for `gpt-3.5-turbo`, and later calls the resulting model identifier. Hosting simplifies the workflow, but it does not give you every decision about training methods, deployment region, or infrastructure. section 5.0 section 5.2.2 section 1.2.2
Resource constraints change the choice of method. Fine-tuning generally needs fewer resources than training from scratch; LoRA reduces the trainable assets to small low-rank matrices. Mixture-of-experts models need extra attention to experts and gating mechanisms. Neither technique excuses weak data: data quality and size remain major determinants of success. section 5.2 section 5.3.4 section 5.5
When training vocabulary meets prompting
DreamBooth can fine-tune an image model from a few example images, but the words used to prompt it still matter. In one avatar project, checking the model’s image-caption dataset revealed why guessed class descriptions produced poor results and suggested wording that worked better. section 7.2.3
Do not fine-tune every time results worsen. First inspect failing cases and changing prompts. A current-events Q&A bot may need fresher retrieval data, while another problem may yield to a prompt change. If fine-tuning is warranted, identify the missing examples and check whether the updated model disrupts downstream prompts. Prompt engineering and fine-tuning can also work together. section 6.3.1 section 7.2.3
