Insecure output handling occurs when people or software trust, interpret, or execute AI-generated content without sufficient scrutiny. In integrated systems, an incorrect output can trigger privilege escalation, remote code execution, data loss, or operational disruption. section 11.1.3
Use a zero-trust process for AI-generated outputs:
- Validate and sanitize every output before execution. section 11.1.3
- Scan for harmful commands such as DELETE, DROP, or UPDATE. section 11.1.3
- Require human approval for high-risk actions, especially database modifications. section 11.1.3
- Test AI-generated queries in sandbox environments before applying them in production. section 11.1.3
The destructive SQL query
An AI system asked to generate a query for retrieving sales data produces a DELETE query instead. Without guardrails and output validation, executing it could remove an entire database. section 11.1.3
