Labeling bias is systematic distortion introduced when annotators’ assumptions affect the labels assigned to data. It is one possible source of bias in an artificial intelligence (AI) pipeline, alongside data collection and deployment. Bias is not automatically harmful: it can reflect a mismatch between training data and the diverse realities the system should represent. section 3.2
The central risk is that subjective or uneven labeling can contribute to outputs that misrepresent populations or reinforce harmful stereotypes. These effects may intersect with differences involving race, gender, class, age, religion, disability, or other personal identity attributes. Examining who labels data, which assumptions shape judgments, and whose experiences are represented helps reveal this risk. section 3.2
