Trustworthiness is the practical confidence that an AI system can be relied upon to behave safely, explain its conclusions, and respect fundamental rights. It matters most when errors affect lives, livelihoods, or access to essential services. section 3.4.2
Deep-learning systems can undermine trust through several weaknesses:
- Adversarial examples can use subtle pixel changes to disrupt surveillance systems, even when attacks are difficult to create. section 3.4.2
- Black-box models may provide conclusions without justifications, creating problems for military decisions, criminal justice, lending, and other high-stakes uses. section 3.4.2
- Risk-based regulation can ban systems threatening safety or livelihoods and impose stringent checks on AI used in critical infrastructure and public services. section 3.4.2
A mission-critical decision
A military operator may need to justify a life-and-death decision made by an AI system. An unexplained conclusion makes reliance difficult, regardless of the system’s efficiency. section 3.4.2
