liveBook · Manning

Equality of opportunity

As explained in AI Model Evaluation

Equality of opportunity is a fairness criterion requiring comparable true-positive or qualified-positive outcome rates across demographic groups.

Equality of opportunity measures whether users who are qualified for a positive outcome receive it at comparable rates across demographic groups. In recommendation systems, qualification usually means having relevant items available, so the metric compares true-positive rates among similarly interested users rather than comparing everyone’s exposure. section 3.2.2

Use the metric to interpret disparities without treating them as automatic proof of unfairness:

  • Compare the proportion of qualified users receiving relevant recommendations for each demographic group. section 3.2.2
  • Prefer this criterion over demographic parity when reliable relevance labels exist and users may have different preferences. section 3.2.2
  • Treat a disparity as a diagnostic signal requiring investigation into preferences, data imbalance, model bias, business rules, and product strategy. section 3.2.2
  • Run the check as part of an offline verification or pre-flight suite before considering an A/B test. section 2.1.2

Movie recommendations

Instead of asking whether all age groups receive blockbuster recommendations at the same rate, ask whether users in each age group who have relevant movies available are equally likely to receive those movies. Unequal true-positive rates identify where deeper evaluation is needed. section 3.2.2

How AI Model Evaluation covers it

How deep it goes
Defines qualified-user true-positive comparisons and their interpretation, but stops short of prescribing a complete statistical testing procedure.
Examples
Uses blockbuster recommendations and age or gender groups to compare relevant-outcome rates.
What it assumes
Assumes demographic groups and reliable relevance or qualification labels, while distinguishing them from general exposure rates.
What you can do after
Diagnose uneven recommendation outcomes and prioritize investigation before experimentation or deployment.
How it connects
Connects demographic parity, relevance labels, diagnostic evaluation, recommendation systems, and model bias.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Model Evaluation. The section links point to the parts of the book it draws on.