Harm is an adverse effect or injury that a generative AI system may cause to individuals, groups, society, or other affected parties. Identifying harm means examining the use case, model capabilities, deployment scenario, and sensitive industry context rather than treating every model as equally risky. section 13.3.1
A practical harm-identification process includes:
- List possible harms and classify them into risk categories such as bias, unfair treatment, privacy concerns, and inaccurate or outdated information. section 13.3.1
- Define each harm precisely by describing its real-world effects and distinguishing areas such as representational, allocational, and capability harms. section 13.3.1
- Rank risks with a rubric for high, medium, and low impact, then use red-teaming and stress testing to expose weaknesses and vulnerabilities. section 13.3.1
Evaluating a customer-service chatbot
For a chatbot, evaluators might test whether responses show gender or racial bias, expose personally identifiable information, or provide inaccurate answers. They first create diverse inputs likely to elicit prioritized harms, generate outputs, document examples, define context-specific metrics, and quantify harmful instances. Manual review validates the approach before automated evaluation is added to an engineering pipeline. section 13.3.1 section 13.3.2
Harm measurement is continuous: repeat evaluations after mitigation, check for regression, and scale testing to risk. Lower-risk systems need less extensive testing, while higher-risk systems may warrant internal and external red-teams. section 13.3.2
