liveBook · Manning

Fairness evaluation

Fairness evaluation is the assessment of whether a model or decision process produces unjustified disparities in outcomes, treatment, or errors across relevant groups.

How 2 Manning books explain it

AI Model Evaluation

A fairness evaluation assesses whether a model's outcomes or error rates differ unfairly across protected or otherwise relevant groups.

How it is used
Uses group comparisons as offline diagnostic checks before experimentation or deployment.
How deep it goes
Defines demographic parity and equality of opportunity, shows disparity calculations, and stops short of prescribing universal fairness thresholds.
Examples
Uses blockbuster and long-tail movie recommendations, group disparity ratios, and unequal image-cropping exposure as examples.
What you can do after
Diagnose potential group disparities and decide which model behaviors require further investigation before launch.

Read how AI Model Evaluation explains fairness evaluation →

AI Governance

Fairness testing evaluates whether an AI system produces unjustified disparities or inequitable outcomes for people or groups.

How it is used
Detects and investigates inequitable outcome differences in LLM-based systems, including systems that cannot be retrained.
How deep it goes
Details paired scenarios, outcome labels, group-rate measurement, and release controls, but leaves harm selection and fairness judgments unresolved.
Examples
Uses ClaimAssist claim decisions, paired sensitive-signal scenarios, group denial rates, and output checks as test examples.
What you can do after
Diagnose disparities by running paired tests, comparing group outcome rates, and escalating threshold breaches for review.

Read how AI Governance explains fairness evaluation →

Also covered in

What to know first

Related concepts

Questions about fairness evaluation

What is fairness evaluation?

Fairness evaluation is the assessment of whether a model or decision process produces unjustified disparities in outcomes, treatment, or errors across relevant groups.

Which Manning books explain fairness evaluation?

AI Model Evaluation and AI Governance.

What should I know before learning fairness evaluation?

Manning books that teach fairness evaluation usually introduce language model first.

What is fairness evaluation a kind of?

Fairness evaluation is a kind of evaluation. Evaluation is the systematic assessment of a system, model, process, or outcome against relevant criteria or objectives using evidence.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.