Data curation prepares fine-tuning data so a specialized model develops the intended domain knowledge, style, and capabilities. It is a decisive stage: data quality, relevance, and comprehensiveness can determine whether fine-tuning succeeds. Fine-tuning sets are smaller than pretraining corpora, so they demand substantially higher quality and closer task fit. section 1.4.2
Choose data sources according to the target task and available resources:
- Use publicly available datasets for broad coverage and established benchmarks, particularly when beginning fine-tuning. section 1.4.2
- Prefer curated, task-specific datasets with quality filtering and deduplication for fine-tuning tasks such as sentiment analysis, named entity recognition, and question answering. section 1.4.2
- Treat large web corpora such as Common Crawl and the Pile as better suited to pretraining than task-focused fine-tuning. section 1.4.2
Figure 1.7 Task-specific fine-tuning process. Pretraining creates a base model that is then further trained to perform multiple, more complex tasks, such as summarization, classification, and code or text generation.

Task-specific fine-tuning builds on a pretrained base model and can specialize it for summarization, classification, or code and text generation. section 1.4.2




