A report-writing assistant highlights a low-confidence passage, but its user ignores it. The warning is visible; its meaning is not. A mental model is the user’s working understanding of what an AI feature does, how it produces results, and when its output needs review. Interface explanations should bring that understanding closer to the system’s actual capabilities and limits. section 10.1.3 section 10.2.2
Shape the mental model around the decisions users must make:
- Learn what users already believe. Ask about AI experience, trust, reactions to errors, confidence in their own work, and desired control. Some may expect human-like intelligence; others may distrust AI after a bad experience. section 10.1.2
- Make the process legible. Signal when AI is working, explain relevant capabilities, and use familiar interactions such as autocomplete or chat where they reduce friction. section 10.2.2
- Show where judgment is still required. Pair uncertainty cues with explanations, source links, and opportunities to verify or edit rather than expecting a score alone to guide action. section 10.2.1 section 10.2.2
- Test the interpretation, not just the interface. Include plausible errors in prototypes, watch whether users act on warnings, and revise explanations when they do not. section 10.1.3
A report draft that teaches its limits
Before generating a report section, the assistant shows the data points and links to their original sources. During generation, a progress window explains what it is doing. If required data is missing, it warns the user before displaying the draft. Paragraph-level confidence scores and explanations then identify passages needing review. Each step helps the user understand the draft as work to check, not a finished answer to accept automatically. section 10.2.1
Table 10.1 Examples of errors you can simulate in your design artifacts
| Error type |
Example |
Observed user behavior |
Design adjustments needed |
| Hallucination |
AI falsely states CO2 emissions dropped 30% instead of 15%. |
Overtrusting users accept AI; skeptics reject AI entirely. |
Add confidence indicators and fact-checking prompts. |
| Regulatory misinterpretation |
AI incorrectly says Scope 3 reporting is optional. |
Legal experts catch it; others miss it. |
Include source citations and human validation steps. |
| Overconfidence in uncertain data |
AI estimates water consumption trends without full data. |
Some trust AI blindly; others demand citations. |
Use confidence scores and “Verify this data” prompts. |
| Formatting and structural problems |
AI-generated tables misalign financial figures. |
Users discard the entire report due to formatting flaws. |
Provide editable report structures for refinements. |
Use the simulated-error table to plan tests: compare how users respond to false claims, regulatory mistakes, uncertain estimates, and formatting failures, then adjust verification cues accordingly. section 10.1.3
