A language model can reproduce unwanted associations found in the language it learned from. A de-biasing technique tries to reduce such patterns. The goal is difficult to pin down: language reflects human perspectives, ideas about bias change across cultures and time, and historical inequalities can be important to represent in some contexts. section 3.2
An early approach
One early method adjusted word embeddings—the representations of words used by a model—so that a word such as “nurse” sat equally far from gendered terms. But removing an association mathematically can also remove meaningful information. Equal distance alone does not establish fairness. section 3.2
Bias mitigation now extends beyond adjusting individual word representations:
- Diversify and document training data, including its demographic coverage and known limitations. section 3.2
- Use human feedback, output filters, fairness-focused evaluation, prompt design, and output audits. section 3.2
Measurement comes before mitigation. Benchmarks such as ToxiGen and BOLD examine toxic language across groups and differences in sentiment associated with demographic attributes. They help identify problems and compare models, but effective mitigation remains challenging; even human judgments can differ. section 10.19 section 3.2
