Bias is a systematic tendency that skews an AI system’s treatment of people. It can look like a working model making consistently worse decisions for a demographic group. A recruiting system trained on historically male-dominated hiring data, for example, learned to penalize résumés mentioning women’s activities or colleges. The failure was in the pattern learned from the data, not a software crash. section 11.0
The harm depends on what the system decides and how widely it is used. Documented failures include unequal criminal-risk flags, harmful image misclassification, and lower care recommendations for black patients when health-care spending stood in for health needs. At production scale, such patterns also create legal, financial, and reputational risk. section 11.1.4 section 11.1.3
Trace and test bias across the data and model layers:
- Inspect training data before fine-tuning. Human decisions and support logs may encode unequal treatment; training on those interactions can reproduce it, while repeated fine-tuning can reinforce it. section 11.2 section 11.2.1
- Compare outputs across demographic groups, including combinations of attributes. In a résumé experiment, varying names exposed racial, gender, and intersectional differences that a single overall result could conceal. section 11.2.3
- Address disparities during training as well as in data. A base model does not guarantee fairness after fine-tuning; control of an open-source training loop permits changes to datasets, evaluation, and loss functions. section 11.3 section 11.3.1
A customer-service fairness check
Suppose an assistant gives shorter replies to customers with some names than to customers with others. Compare response quality across groups before using the conversations for further fine-tuning. One LoRA training sketch adds a demographic-gap penalty to cross-entropy loss, with a weight that trades a stronger fairness constraint against possible accuracy loss. Treat that sketch as an illustration, not a validated fix: as written, it averages each group’s softmax probabilities into a single scalar, which may fail to measure the intended difference between groups. section 11.2.1 section 11.3.2
Here, bias means unequal system behavior—not a neural-network bias parameter. The latter is an offset in an operation such as output = input × weight + bias; LoRA can leave those parameters unchanged with `bias="none"`. That setting does not establish demographic fairness. section 5.3.4 section 11.3
