Implicit bias is an unintended or unconscious association or preference that systematically influences judgments or behavior. In LLMs, it can appear as harmful patterns that users did not explicitly request. section 9.1.1
Treat implicit bias as a deployment risk, not merely a benchmark score:
- Training data can produce latent bias that appears in unexpected prompts and real-world uses. section 9.1.1
- Explicit bias benchmarks may miss serious harms, including dialect-based discrimination and debunked race-based medical recommendations. section 9.1.1
- Evaluate the proposed use, affected groups, and surrounding safeguards before deploying a model; some uses may be unsuitable without adequate mitigations. section 9.1.1
Concrete failures
Observed examples include worse treatment of African American Vernacular English and recommendations for debunked race-based medical practices when advising on care for different races. section 9.1.1
