liveBook · Manning

Bias

As explained in AI Governance

A systematic distortion in data or model behavior that can produce unfair or skewed outcomes.

A claims model can cite the correct policy clause and still deny some ethnic groups more often or blame older drivers despite evidence of shared responsibility. Algorithmic bias is a systematic difference in outcomes driven by characteristics that should not determine the result. Security and privacy safeguards alone do not prevent it. section 6.2 section 6.0

Historical patterns can make unfair outcomes look predictive. A hiring system trained on predominantly male applicants downgraded résumés containing terms associated with women. A hospital risk algorithm used healthcare spending as a proxy for need, assigning Black patients lower priority because they had historically received less care. These patterns need not reflect individual responsibility or actual need. section 6.2.1 section 6.2

Figure 6.1 Stable Diffusion perpetuates gender stereotypes by returning only male engineers when prompted with “a color photograph of an engineer”

The image-generator output illustrates how a job prompt can reproduce gender stereotypes. section 6.0

Test for bias before deployment and correct disparities rather than assuming historical data is fair. One useful comparison holds the substantive case constant while varying demographics: studies have compared identical résumés with different names and emergency cases with the same symptoms but different patient backgrounds. Training on AI-generated outputs can also reduce coverage of rare or minority cases, making those groups especially vulnerable. section 6.2.1 section 6.0

Autonomous agents can amplify biased decisions by acting rapidly across many cases. For high-impact actions, governance options include human approval, decision logs for audit, and strict domain constraints. These controls support oversight but do not make a model's stated reasoning reliably explain its decisions. section 2.6.4

How AI Governance covers it

How deep it goes
Defines algorithmic bias and traces historical-data and proxy failures, while recommending testing and correction without specifying a detailed mitigation procedure.
Examples
Uses claims decisions, résumé rankings, hospital prioritization, and occupational image generation to illustrate unequal outcomes.
What you can do after
Evaluate and understand bias risks by comparing outcomes across demographic variants and considering oversight for high-impact decisions.
How it connects
Connects bias with fairness, proxy variables, explainability, model collapse, and autonomous agents.

Also explained in

Compare how 38 books explain bias →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Governance. The section links point to the parts of the book it draws on.