liveBook · Manning

Monitoring plan

As explained in AI Governance

A documented plan specifying how post-market information about an AI system will be collected and evaluated.

A post-market monitoring plan specifies how a provider will watch a particular AI system in production. It belongs in the system’s technical documentation; unlike a quality management system (QMS), which describes how the organization builds AI, the plan describes how that system is monitored. section 7.5.1

Make the plan operational by specifying:

  • What data to collect, such as denial rates by claim category, adjuster override rates, and accuracy scores against a periodically refreshed test set. section 7.5.1
  • How often to review it, such as weekly automated dashboards and monthly human review. section 7.5.1
  • Which changes trigger investigation, such as spikes in denials or overrides, or drops in accuracy; define corrective actions, such as suspending automated decisions for an affected category and escalating to the QMS. section 7.5.1

Providers design the monitoring system. Deployers carry out its oversight measures, retain automatic logs, and share relevant observations with the provider; they also monitor local circumstances the provider cannot account for, such as jurisdiction-specific requirements and affected populations. section 7.5.1

How AI Governance covers it

How deep it goes
Details metrics, review cadence, investigation thresholds, and corrective actions, while leaving deployer-specific obligations to separate requirements.
Examples
Uses ClaimAssist V5 denial rates, adjuster overrides, quarterly refreshed accuracy scores, review schedules, and intervention thresholds.
What you can do after
Configure a system-specific monitoring workflow with data, review intervals, trigger thresholds, and corrective actions.
How it connects
Connects post-market monitoring with technical documentation, quality management systems, and deployer oversight.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Governance. The section links point to the parts of the book it draws on.