Testing AI products means examining both how people interpret uncertain AI behavior and how deployed outputs affect user expectations and product outcomes. Prototype tests can reveal issues before launch; production testing provides evidence from real inputs and use, but a shaky early release can drive users away. section 5.4.4 section 10.1.3
A practical testing approach can include:
- Use prototypes or code-based artifacts, and make AI outputs realistic enough to reflect uncertainty and possible errors. section 10.1.3
- Observe users’ responses to errors; tailor examples to their context when testing personalization, and consider Wizard of Oz to simulate AI behavior. section 10.1.3
- When the product is mature enough for a live release, collect direct feedback and connect model outputs to satisfaction, engagement, retention, and conversion. section 5.4.4
- Keep testing and learning cross-functionally, aligning technical and domain measures with user-centered goals. section 12.1
Table 10.1 Examples of errors you can simulate in your design artifacts
| Error type |
Example |
Observed user behavior |
Design adjustments needed |
| Hallucination |
AI falsely states CO2 emissions dropped 30% instead of 15%. |
Overtrusting users accept AI; skeptics reject AI entirely. |
Add confidence indicators and fact-checking prompts. |
| Regulatory misinterpretation |
AI incorrectly says Scope 3 reporting is optional. |
Legal experts catch it; others miss it. |
Include source citations and human validation steps. |
| Overconfidence in uncertain data |
AI estimates water consumption trends without full data. |
Some trust AI blindly; others demand citations. |
Use confidence scores and “Verify this data” prompts. |
| Formatting and structural problems |
AI-generated tables misalign financial figures. |
Users discard the entire report due to formatting flaws. |
Provide editable report structures for refinements. |
Use these simulated failures to prompt observation and design decisions: users may accept or reject hallucinations, miss regulatory errors, demand evidence for uncertain estimates, or discard reports with formatting flaws. section 10.1.3
Testing also belongs in ongoing AI governance: regular bias testing and retraining on diverse datasets help guard against reinforcing existing biases. section 11.6
