An AI system is reliable when it gives accurate, factual answers, behaves consistently, acknowledges when it cannot succeed, and stays grounded in verifiable information. It must also act fairly, meet latency and cost requirements, take safe actions within defined boundaries, and maintain quality as models and data change. Uptime alone is not enough: a system can remain available while confidently giving wrong answers. section 1.3.3
Benchmark capability is not production reliability. Top models resolved more than 85% of problems on SWE-bench Verified but fell to 58–65% on SWE-bench Pro’s fresh, unfamiliar codebases. Strong test results therefore do not remove the need to evaluate the conditions a deployed system will actually face. section 1.1
Engineer reliability across the system, not just the model:
- Outputs: shape behavior through prompting and ground answers in verified information through retrieval. Check for fabrication rather than treating a fluent response as evidence. section 1.3.3 section 1.0
- Actions: give agents defined permissions and account for reversibility, tool failures, and failure containment. Answer quality alone is insufficient when a system can act. section 1.2.4
- Operations: evaluate the deployed system and monitor quality as data drifts or models update. Make accuracy–latency and safety–cost trade-offs deliberately. section 1.3.3 section 1.6
A multipart customer request
A customer sends a boot photo, asks for a similar waterproof product, then asks about returns. A coordinated system routes image understanding, product search, and policy retrieval to specialized agents before combining their results. Separate agents can be tested and improved independently; if one has trouble, the others can continue rather than forcing the entire system to fail. section 8.0 section 8.1.7
Predictable behavior is not the same as correctness. In health care, even a model that scores 91% on medical-licensing exams may be brittle on novel cases. Reliability requires checking what the whole system does under its intended operating conditions, especially where an incorrect answer or action can cause harm. section 5.2.1 section 1.3.3
