A visible error, an invisible error
A sentiment model may label a strongly negative movie review as positive. Put the score beside the review and a user can spot the mistake; aggregate thousands of scores without showing individual results and the same mistake may go unnoticed. section 10.2.2
A failure mode is a distinct way an AI system can produce an unwanted result. Predictive AI can make false-positive or false-negative predictions. Generative AI can confidently make things up. Agentic AI can make those errors and compound them across multiple steps, including planning and selecting what to do next. section 10.2.2
Before designing how an interface handles failure, ask:
- What kinds of errors can this system make? section 10.2.2
- Which errors will users recognize, especially when an output conflicts with what they know or appears beside its input? section 10.2.2
- How much error can users tolerate in this situation? Consider the stakes as well as the user. section 10.2.2
Not every technical mistake has the same effect: an unrealistic brainstorming suggestion might spark a useful idea. Assess the consequence of an error, not just whether it occurred. section 10.2.2
