liveBook · Manning

Fairness evaluation

As explained in AI Governance

Fairness testing evaluates whether an AI system produces unjustified disparities or inequitable outcomes for people or groups.

If ClaimAssist denies one group’s claims more often than another’s for the same claim type, that disparity needs investigation. Fairness evaluation tests observed system behavior for inequitable outcomes, including when a hosted LLM cannot be retrained. section 6.2.3

A repeatable evaluation loop can:

  • Select critical journeys, such as denials, escalations, fraud flags, and summaries; create paired scenarios that differ in one sensitive signal. section 6.2.3
  • Capture prompt and attribute variants, then reduce outputs to labels such as approve, deny, escalate, or require human review. section 6.2.3
  • Compare outcome rates by group and investigate material differences, such as unequal denial rates for the same claim type. section 6.2.3
  • Rerun tests after prompt or data changes. Specific prompt instructions create verifiable expectations, but monitoring and testing are still needed to catch failures. section 6.2.3 section 6.2.4

ClaimAssist in production

A scheduled CI job runs tests nightly and when prompts or retrieval sources change. LangFair structures runs and computes metrics; a dashboard supports weekly review. A warning-threshold breach blocks release and triggers manual bias review. A green dashboard is not proof of fairness: chosen groups, thresholds, or scenarios may miss important harms. section 6.2.3

Figure 6.5 Bias Testing Workflow for LLMs. A step-by-step process: generate prompts, collect LLM outputs, structure data, and analyze for bias using fairness toolkits.

Use the workflow diagram to follow the sequence from prompt generation through output collection and analysis. section 6.2.3

Treat measurements as evidence to review, not a final fairness verdict. Check reference data for biased patterns; where sensitive attributes are available, protect them in storage for group monitoring while excluding them from model inputs. Review outputs for concerning patterns and route high-stakes decisions, such as fraud flags or denial letters, to a human. section 6.2.4

How AI Governance covers it

How deep it goes
Details paired scenarios, outcome labels, group-rate measurement, and release controls, but leaves harm selection and fairness judgments unresolved.
Examples
Uses ClaimAssist claim decisions, paired sensitive-signal scenarios, group denial rates, and output checks as test examples.
What you can do after
Diagnose disparities by running paired tests, comparing group outcome rates, and escalating threshold breaches for review.
How it connects
Links fairness testing to large language models, CI, prompt controls, data monitoring, and human review.

Also explained in

Compare how 2 books explain fairness evaluation →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Governance. The section links point to the parts of the book it draws on.