Safety is the property of an AI system's behavior that avoids causing or enabling harm. For language models, it includes preventing unwanted outputs while recognizing that strict refusal can reduce usefulness. section 3.5 section 3.6
AI safety addresses actions creators do not want models to take. AI alignment narrows the focus to matching a system's goals with its creators' intentions or human values. section 3.5
Common safety measures and concerns include:
- Policies and safeguards restrict racist or extremist content, unsafe advice, and instructions for illegal or harmful actions. section 3.5
- Post-processing classifiers can block violative responses, but heavy-handed filtering may create false positives and frustrate users. section 3.6
- Autonomous agents require failure anticipation, side-effect mitigation, monitoring, sandboxed tools, policy guardrails, and meaningful human control. section 8.25
- Physical agents generally retain human authority over mission-critical and safety-sensitive decisions. section 8.8
Helpful versus harmless
A model may comply with a request in a way that appears helpful but produces sexist or otherwise harmful text. Refusing more aggressively lowers that risk, yet may also reject benign questions. Safety therefore involves balancing helpfulness, harmlessness, and honesty. section 3.6
Figure 3.1 The logic of a response decision with a post hoc toxicity classifier
A decision flow shows how a post hoc toxicity classifier can determine whether a generated response is released or blocked. section 3.6
