A task-specific dataset is a collection of examples selected and prepared for training or evaluating a model on a particular task. In enterprise fine-tuning, its quality, representativeness, and fit to the target domain often matter more than raw volume. section 9.1.2
Prepare the dataset with these practical considerations:
- Clean and normalize the data, including tokenization where appropriate. section 9.2.1
- Label examples and convert them into the input format required by the language model. section 9.2.1
- Represent the target task, domain, and range of scenarios expected in production. section 9.2.1
- Start with a baseline dataset, then adjust its size and composition using performance monitoring and comparative evaluation. section 9.1.2
- Balance quantity against quality, computational resources, time, and the complexity of the task; a starting point may range from several hundred to several thousand examples. section 9.1.2
Baseline dataset iteration
An organization begins fine-tuning with a baseline collection of task examples, preprocesses and labels it, and monitors validation and test performance. It expands or revises the dataset when results indicate underfitting, overfitting, or insufficient coverage. section 9.1.2 section 9.2.1
