An adversarial test set contains challenging or ambiguous inquiries likely to trigger hallucinations. It complements a generic test set of common customer questions, probing failure cases rather than mirroring typical production traffic. section 9.1.1
Build the set with your team around inquiries that could lead an agent astray:
- Complex product comparisons that may prompt invented features. section 9.1.1
- Edge cases involving discontinued products or limited-time offers. section 9.1.1
- Questions with multiple valid interpretations or requests combining multiple topics. section 9.1.1
Use the inquiries to evaluate the system, then validate factual claims in its responses against grounding data. Grounding facts need clear sources so claims can be checked precisely. section 9.1.1
