Diversity is the variation among examples in a dataset. In evaluation, its value depends on how well the examples cover the input conditions relevant to the task, not simply on the number of examples collected. section 4.3.2 section 4.3
Medical advice chatbot
A useful evaluation set would span symptoms, demographics, question types, severity levels, and clinical contexts. Five thousand repetitive or trivial examples may reveal less than a well-balanced set of one thousand varied cases. section 4.3.2
The diversity needed depends on task variability and the questions the evaluation must answer. Open-ended dialogue, multi-step reasoning, and planning vary more than narrow tasks such as spam filtering. Data needs also grow when inputs, outputs, evaluation criteria, or subgroups vary across several dimensions. section 4.3.2
When choosing or generating evaluation examples, consider:
- Coverage: Which input conditions and subgroups matter for the intended evaluation? A large but repetitive dataset can miss important cases. section 4.3.2
- Metric stability: Do small changes to the dataset cause large score swings? Confidence intervals and repeated sampling can reveal uncertainty, but a stable score alone does not describe which conditions are covered. section 4.3.2
- Consequences of mistakes: In synthetic data, favor variety when missing a genuine weakness is more consequential than a false alarm; favor correctness and plausibility when false alarms could mislead or prompt unnecessary remediation. section 4.4.3
Table 4.6 Prioritizing variety versus correctness in synthetic evaluation data generation.
|
Goal
|
Priority
|
Rationale
|
|
Discover
hidden failure modes
|
Variety
|
Broad
coverage increases the likelihood of exposing unexpected weaknesses and
reveal system limits.
|
|
Explore
emergent behaviors or long-tail scenarios
|
Variety
|
Rare or
unusual cases are unlikely to appear in highly realistic datasets.
|
|
Debug
specific issues
|
Variety
|
Multiple
variations help isolate and understand failure patterns.
|
|
Estimate
real-world performance
|
Correctness
and plausibility
|
Evaluation
data should resemble actual user behavior and deployment conditions.
|
|
Evaluate
fairness
|
Correctness
and plausibility
|
Unrealistic
examples can distort subgroup comparisons.
|
|
Measure
factual accuracy or correctness
|
Correctness
and plausibility
|
Reliable
labels and realistic scenarios are essential.
|
|
Support
stakeholder, audit, or deployment decisions
|
Correctness
and plausibility
|
Evaluation
cases must be defensible and representative.
|
Compare priorities for evaluation goals: broad variety can help uncover hidden failures, while realistic scenarios and reliable labels matter for estimating real-world performance or evaluating fairness. section 4.4.3









