Bias is a systematic tendency in data or model behavior that can produce distorted or unfair outputs. A generative AI model may reflect biases in its training data, including stereotypes. In an enterprise application, those outputs can create ethical, legal, and business risks. section 1.5 section 6.7
Check for bias across the application, not just in the model:
- Examine the data, model assumptions, and algorithmic design for possible sources of bias. section 13.1
- Review prompts and generated responses for amplification or propagation of existing biases. section 6.7
- Consider retrieval from diverse sources when using retrieval-augmented generation (RAG); it can help mitigate bias inherited from training data, but does not replace output review. section 8.8 section 1.5
Reviewing an enterprise chatbot
If a chatbot response reinforces a stereotype, inspect both the prompt and the information available to the model. Treat the response as a fairness concern before it reaches users. For consequential decisions, keep a person responsible for the final decision rather than letting the AI act alone. section 1.5 section 6.7 section 13.1
Bias is not solely a data-cleaning problem: technology, society, and policy interact. Responsible deployment calls for detecting and correcting bias, with governance and human oversight matched to the potential harm. section 13.1 section 13.0
