liveBook · Manning

Underrepresentation

As explained in Machines that Think

Underrepresentation is the insufficient presence of a group or its examples in data, systems, or decision processes relative to the relevant population.

Underrepresentation occurs when some groups, perspectives, languages, or experiences are sparsely represented or absent from data used by language technologies and decision systems. What is rarely recorded leaves little statistical trace, so absence can distort how groups and experiences appear. section 7.3.4

Large language models often draw on high-resource languages and internet-accessible discourse. When prompts concern less represented contexts, models may rely on dominant narratives, flatten distinctions, or omit locally grounded knowledge. Historical training data also limits coverage of later events and developments. section 7.3.4

Sparse or uneven representation can contribute to unequal system outcomes:

  • A healthcare algorithm used cost as a proxy for medical need, underestimating Black patients’ needs against a history of less care for the same level of illness. section 7.3.5
  • Commercial speech-recognition systems had higher word error rates for Black speakers; dialect-insensitive hate-speech systems risked over-moderating African American English. section 7.3.5
  • Gender-classification systems performed worse on darker-skinned women, while predictive-policing feedback loops could concentrate surveillance in already over-policed neighborhoods. section 7.3.5

How Machines that Think covers it

How deep it goes
Traces how sparse representation can shape model responses and system outcomes through several concrete cases.
Examples
Uses healthcare-cost prediction, speech recognition, gender classification, dialect moderation, and predictive policing examples.
What you can do after
Evaluate how sparse representation can distort outputs and contribute to unequal treatment across groups.
How it connects
Links underrepresentation to representational harm, training data, proxy variables, and feedback loops.

Also explained in

Compare how 2 books explain underrepresentation →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machines that Think. The section links point to the parts of the book it draws on.