Validation checks generated code, model behavior, data, and outputs before they are trusted in an enterprise application. It is not a one-time approval: generated code needs human review before production, while deployed models and data pipelines need regular testing and monitoring. section 5.3.4 section 11.1
Put checks where a failure could affect the application:
- Review and test generated code for bugs, security vulnerabilities, performance, readability, maintainability, and compliance with standards and regulations. section 5.3.4
- Validate data sources and pipelines for accuracy, task relevance, and freshness; monitor them and document data lineage. section 11.1
- Test model performance regularly and validate outputs during postprocessing, alongside formatting, content classification, and quality assurance. section 11.1 section 10.6
- Check input and output validity at security boundaries. Pair input validation with rate limiting and resource monitoring to help protect model availability. section 13.2.2 section 13.2.3
Generated SQL is not an approved query
Suppose an application sends a request for a report to an LLM, then runs the returned SQL directly. An attacker can shape the request so the generated query contains a destructive command. Do not treat syntactically valid output as safe: check input and output validity, use prepared statements, enforce database permissions, and conduct audits. section 13.2.2
Keep people in the loop. Tool choice and documented generation guidelines can improve consistency, but neither replaces verification. Feedback on tool problems and monitoring in production help keep checks useful as tools, data, and application behavior change. section 5.3.4 section 11.1
