Representational harm occurs when language technologies systematically underrepresent, erase, stereotype, or misdescribe groups and experiences. Bias can arise not only from skewed associations, but also from missing perspectives: what is rarely recorded leaves little statistical trace. section 7.3.4
When training data sparsely represents a language or cultural context, a model has fewer detailed examples to draw on. Its responses may flatten distinctions, omit locally grounded knowledge, or fall back on more dominant narratives. Historical training data can also leave later events and developments outside the model’s learned patterns. section 7.3.4
Representational harm differs from allocative harm, which concerns how systems distribute opportunities, resources, or services. The harms can interact: misrepresentation or uneven recognition may shape decisions about interviews, healthcare, education, or institutional support. section 7.3.5
Examples show how gaps in representation can produce unequal system behavior:
- A healthcare algorithm underestimated Black patients’ needs when it used healthcare costs as a proxy for medical need. section 7.3.5
- Commercial speech-recognition systems had higher word error rates for Black speakers, while gender-classification systems performed worse on darker-skinned women. section 7.3.5
- Hate-speech detection could mistake African American English for toxicity when data and annotation did not account for dialect. section 7.3.5
