10 Evaluating agents
This chapter covers
- Why agent evaluation must be automated
- Concrete procedures for automating evaluation
- How to put automated evaluation to work in practice
An agent goes beyond merely suggesting an answer. It decides for itself what information to seek, which tools to use, in what sequence to execute tasks, and when to stop. This autonomy is a powerful advantage, but a single small mistake can cascade into real-world costs and risks: payments, procurement, permission changes, and external communications. That is why rigorous evaluation is essential: to prevent agents from causing unexpected harm and to protect users from potentially significant losses.
Yet evaluating agents is truly difficult. Agent evaluation must be automated because the evaluation space is enormous. An agent’s output is not the result of a single operation; it is formed through a chain of multistep reasoning and actions that interlock and accumulate. If you evaluate only the final output, it becomes difficult to distinguish a correct result that happened by luck from one produced by consistently sound reasoning. Proper evaluation requires examining intermediate reasoning, tool selection, and action decisions. As autonomy increases, the number of possible action paths expands, and with it, the number of failure points and evaluation dimensions grows exponentially. Without automation, evaluation becomes the bottleneck that blocks development and improvement.