liveBook · Manning

Override rate

As explained in AI Governance

An override rate is the frequency with which users reject or change an AI system's suggestion.

An override rate tracks how often reviewers reject or change an AI suggestion. Alongside edits and decisions overturned on appeal, it helps monitor whether people remain engaged with AI-assisted decisions. section 6.3.1

Compare production with testing

If controlled testing shows a 20% correction rate but production reviewers reject suggestions only 5% of the time, the gap signals possible overreliance. Compare rates with testing, and track appeals; declining overrides or rising appeals should prompt updates to training and workflow. section 6.3.1

Use the metric as part of an operational feedback loop:

  • Route samples of high-risk cases, including large denials or decisions made with minimal edits, to a second reviewer or QA team. section 6.3.1
  • Provide a low-friction way for reviewers to flag disagreement and briefly record why, with a defined escalation path to the AI team or vendor. section 6.3.1

How AI Governance covers it

How deep it goes
Defines production signals and response steps, but does not specify a formula for calculating the rate.
Examples
Compares a 20% controlled-test correction rate with 5% production rejection and recommends reviewing high-risk cases.
What it assumes
Assumes a controlled-test correction baseline and records of production reviewer decisions or appeals.
What you can do after
Diagnose possible overreliance by comparing production overrides with controlled-test correction rates and tracking appeals.
How it connects
Connects human oversight with AI calibration, reviewer training, and feedback loops.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Governance. The section links point to the parts of the book it draws on.