An override rate tracks how often reviewers reject or change an AI suggestion. Alongside edits and decisions overturned on appeal, it helps monitor whether people remain engaged with AI-assisted decisions. section 6.3.1
Compare production with testing
If controlled testing shows a 20% correction rate but production reviewers reject suggestions only 5% of the time, the gap signals possible overreliance. Compare rates with testing, and track appeals; declining overrides or rising appeals should prompt updates to training and workflow. section 6.3.1
Use the metric as part of an operational feedback loop:
- Route samples of high-risk cases, including large denials or decisions made with minimal edits, to a second reviewer or QA team. section 6.3.1
- Provide a low-friction way for reviewers to flag disagreement and briefly record why, with a defined escalation path to the AI team or vendor. section 6.3.1
