Robustness is the ability of a system to maintain acceptable performance across specified circumstances and perturbations. Characterizing and testing behavior under those circumstances supports acceptance to service. section 2.6
Treat robustness as part of operational safety:
- Record possible failures in a risk register and use explicit evaluations to identify causes and mitigations. section 2.6
- Specify post-development monitoring so system behavior is surfaced, recorded, and escalated through alerts and notifications. section 2.6
- Document who may be harmed when the system functions as intended and when it fails, then decide whether deployment is safe. section 2.6
Impact pairs
Generative art may displace artists or create harmful images; generative text may displace journalists or flood the internet with nonsense; facial recognition may identify dissidents or wrongly identify innocent people as criminals. section 2.6
Figure 2.3 A layered model of ML safety (adapted from Hendrycks et al [7])
Use the layered ML safety model to connect robustness testing with broader safety controls. section 2.6
