Synthetic data consists of artificially generated examples used to supplement a dataset. For model adaptation, a teacher model can expand a small set of high-quality real examples into labeled variations, especially for rare scenarios that are difficult to collect. Generated examples need verification before they are mixed with real data. section 3.4.3 section 3.9
Start with clean human-labeled examples: they anchor the teacher’s output to the desired style, vocabulary, and domain. Without those references, generated examples can reflect the base model’s patterns rather than the target domain. Use synthetic generation when evaluation reveals a coverage gap, not simply to increase volume. section 3.1 section 3.4
A practical generation pipeline follows six steps:
- Load high-quality seed examples and group them by category. section 3.6.1
- Build style-anchored prompts using real examples from the target category, then call a teacher model for each category. section 3.4.3 section 3.9
- Apply an LLM-as-judge quality gate for fidelity, format, and domain safety. section 3.9
- Check distribution alignment using centroid cosine similarity, then mix accepted examples with real data. Use a maximum of 30% synthetic data as a first-run heuristic. section 3.9 section 3.4.3
Judge quality against the task. An invented hostname may be harmless filler in a ticket-classification example, but a summary that invents a host or root cause teaches the confident error the system should avoid. Synthetic data complements rather than replaces real examples; too much can narrow the model’s picture of the real distribution. section 3.4.3
Expanding sentiment examples
A demonstration loads 50–60 examples from the AllAgree split of Financial PhraseBank, where all 16 annotators agreed on each label. It groups seeds by sentiment so a prompt for negative examples can show three real negative examples rather than random samples. The public dataset demonstrates the pipeline without exposing a private support queue. section 3.6.1
Table 3.2 Different data acquisition strategies and what is best used for
|
Strategy
|
Relative cost
|
Label quality
|
Best used for
|
|
Human
annotation
|
High
|
Highest
|
Seed data,
evaluation sets, high-stakes edge cases
|
|
Weak
supervision
|
Low
|
Moderate
|
High-volume
categories with clear labeling rules
|
|
Synthetic
generation
|
Medium
|
High (with
gate)
|
Rare
categories, gap-filling, controlled variation
|
|
Production log
mining
|
Low
|
Variable
|
Distribution
alignment, temporal freshness
|
Compare synthetic generation with human annotation, weak supervision, and production log mining by cost, label quality, and best use. section 3.4
Combine acquisition methods according to the budget and quality target: human labels provide anchors and evaluation examples, weak supervision scales clear rules, and production logs help maintain distribution alignment. Scrub personal data before it reaches annotators or training infrastructure; removing it from a dataset after training does not remove it from the model. section 3.4 section 3.9























