Years of support tickets can look like a ready-made training set until categories drift, severity fields are empty, and automated closures masquerade as resolutions. Data quality determines whether those records can support reliable model adaptation, not just whether they are available in sufficient volume. section 3.0
Adaptation data should contain correct, complete task-level responses, reflect the intended production distribution, and retain traceable origins. Models can learn errors and inconsistent conventions along with useful patterns; a larger noisy dataset is not necessarily better than a smaller curated one. section 3.9 section 6.7.1
Use a repeatable quality checklist before training and whenever the dataset changes:
- Review responses for factual correctness, completeness, and consistent formatting. Include the context needed to answer; when information is missing, demonstrate clarification rather than guessing. section 5.1.7 section 6.7.1
- Cover varied inputs, users, scenarios, and difficulty levels. Match expected response types and difficulty where practical, but deliberately overrepresent rare, high-cost cases such as safety refusals. section 5.1.7 section 6.7.1
- Measure agreement among annotators and improve labeling guidelines when judgments differ. Keep evaluation data separate from training and test regularly on unseen, current queries. section 5.1.7 section 5.10
- Scrub personal data before examples reach annotators, annotation platforms, or training infrastructure. Review indirect identifiers as well as structured identifiers; removing data later does not remove it from a trained model. section 3.9
Choose acquisition methods for their strengths: human annotation is valuable for seed and evaluation data, weak supervision scales labeling, and production logs provide realistic, fresh inputs. Synthetic examples can fill rare categories, but need a quality gate and distribution checks; a maximum 30% synthetic mix is a heuristic, not a guarantee. section 3.9
IT support fine-tuning
A curated support dataset combines practitioner-written Stack Exchange questions and accepted answers with a small general-knowledge slice. Answers follow a one-line summary and numbered steps, source URLs are recorded, and length filters remove truncated or excessively long examples. A 450-example training subset supports fast iteration, while production use calls for broader checks including agreement, bias, and privacy. section 5.1.1
Filtering clinical reasoning
In a healthcare adaptation, quality filtering removed 8% of examples that would have introduced inconsistent reasoning patterns. Early unfiltered experiments sometimes produced confident but internally contradictory reasoning, illustrating why final-answer correctness alone is insufficient. section 6.12.6














