Red-teaming deliberately challenges a generative AI system to uncover vulnerabilities, biases, and harmful behavior that standard testing may miss. It probes both hostile attacks and harms arising from ordinary use. Its job is to discover where to investigate further—not to measure how common a harm is or replace mitigation. section 13.4
A practical enterprise flow has three stages:
- Plan: assemble testers with varied expertise, identify risks relevant to the use case and each model, and decide what data and consistent identifiers to record. section 13.4 section 13.3.1
- Test: challenge the base model and the application, including their safety systems and filters. Adapt the probes to the application’s context. section 13.4
- Follow through: document failures, define metrics and test sets, implement mitigations, and repeat evaluation to check effectiveness and regression. section 13.4 section 13.3.2
Customer-service chatbot
For a customer-service chatbot, testers could probe for unequal treatment across groups, exposure of personal information entered by users, and inaccurate answers. Record the inputs and outputs, then assess harmful results against definitions and metrics specific to that deployment. A discovered failure is evidence of a risk, not an estimate of its prevalence. section 13.3.1 section 13.3.2 section 13.4
Choose tools to match the question: TextAttack can generate adversarial inputs; AI Fairness 360 and TensorFlow Fairness Indicators support bias audits; LIME and SHAP help investigate outputs. Higher-risk systems warrant more extensive testing, including internal and external red teams where feasible. Share findings through compliance channels and test mitigations again rather than treating a passed probe as proof of safety. section 13.4.2 section 13.3.2
