Bias is a systematic pattern in data, rules, or decisions that disadvantages particular groups. In analytics, it matters whenever a report or automated system influences how people are evaluated or treated. Apparent objectivity does not guarantee a fair outcome. section 8.0 section 8.2.2
Hiring from historical resumes
A resume-ranking model trained on past applicants learned from a pool heavily weighted toward men. It gave lower scores to applicants who attended women’s colleges or mentioned “women.” Attempts to remove the gender bias failed, and the project was abandoned. The case also exposes a harder question: recruiters and teams do not share one measure of candidate quality. section 8.0
Bias is not limited to machine learning. A rules-based system can have unintended effects when its designers miss how a rule applies to people; a learned system can reproduce patterns in its training data. Neither approach should be assumed fair merely because it applies rules consistently or handles many records. section 8.2.2
Customer-support triage
A support team considers rules-based and supervised models to assign ticket types and priority levels. Because those labels determine queue order and response speed, a categorization system that streamlines work could also have an unanticipated negative effect on certain customer groups. Accuracy alone does not settle the question of impact. section 8.2.2
Treat biased output as a decision risk, not just a technical defect. Ask whose data shaped the system, which groups may be affected, and whether the resulting decisions can be trusted. The concern extends to large language models trained on human data; biased output has been noted in consequential areas such as healthcare. Automated hiring tools may also face bias-audit requirements. section 8.0 section 8.4 section 12.4.2
