Imagine two identical resumes receiving different rankings because one has a white-associated name and the other a Black-associated name. This is an example of bias: disparate model outputs across personal identity attributes, such as race, gender, class, age, or religion. Here, bias does not mean a neural network’s bias term or the statistical difference between an average prediction and a true value. section 10.7 section 1.12
Large language models learn patterns from text. When their training data contains stereotypes or reflects historical inequalities, their outputs may repeat those patterns. Uneven representation of communities can also produce misleading results. Bias can enter elsewhere, too: annotators may bring assumptions to labeling, and a deployed system may serve some populations better than others. Changing social views can leave older training data out of step with current discourse. section 3.2 section 1.12
A hiring disparity
In a 2024 audit of resume-screening models, large language models ranked identical resumes with white-associated names above those with Black-associated names in more than 85% of cases. A model used to support decisions can therefore reinforce an existing inequality rather than merely reproduce offensive language. section 10.7
Not every mismatch between data and the world is inherently harmful, but stereotypes and unequal treatment can cause real harm in hiring, health advice, and customer service. Post-training and other controls aim to reduce unsafe outputs; bias mitigation remains imperfect, particularly in open-ended or long-context use. Assessing bias means asking whose experiences the data represents and whether model outputs differ across groups. section 3.2 section 10.7 section 3.0
