liveBook · Manning

Evaluation

As explained in AI Agents in Action, Second Edition

Evaluation is the systematic assessment of an agent's output or behavior against defined criteria or expected results.

An agent can give a correct answer while making several wrong tool calls. Evaluation checks both the answer and the behavior that produced it against defined criteria, making failures visible and giving teams a basis for improvement. Evaluation alone cannot fix a poorly architected agent. section 7.2 section 7.0

A test set can specify questions, expected answers, and known-bad answers. For less exact responses, a rubric can describe criteria such as factual accuracy, grounding in retrieved context, tone, and structure. Benchmark tests, grounding checks, human feedback, and evaluator agents answer different questions about agent behavior. section 7.2 section 7.0

A useful evaluation looks beyond the final response:

  • Check which tools the agent selected and whether its response stayed grounded in retrieved context. section 7.2
  • Inspect reasoning steps, the number of steps, latency, and cost. section 7.2
  • Test how the agent behaves when something fails, as well as whether the final answer meets the rubric. section 7.2

Start with a small benchmark

A retrieval-augmented generation (RAG) agent can be tested with questions about a small knowledge collection. For example, a benchmark asks what a fictional engine spins: the expected answer contains “antimatter,” while “fusion” is a known-bad answer. An initial test uses string matching against the expected answer. This is a starting point, not a complete check of the agent’s tool use or grounding. section 7.2.2 section 7.2

In test-driven agent development, define goals and benchmarks first, then test, adjust the prompt or tools, refactor, and repeat. Expect initial failures. Because language-model outputs vary, run tests multiple times to distinguish consistent behavior from a one-off result; a practical starting rule is to run each passing benchmark three times. Make the smallest change that addresses a failure before adding more instructions or tools. section 7.2 section 7.2.2

Figure 7.2 The human process of developing agents using TDAD: first, we think of goals and benchmarks; then we test, develop the prompt, refactor, and repeat the process.

The diagram shows the repeated cycle from goals and benchmarks through testing, prompting, and refactoring. section 7.2

For ongoing use, maintain curated test sets and automated checks, including LLM-as-judge checks, and run them when prompts or models change. Trace prompts, tool calls, token counts, latency, and outcomes; collect human feedback to guide further improvements. A minimal setup can start with basic logging, tests, and guardrails. section 11.1.5

Use offline tests as an initial release gate, followed by shadow traffic, a small canary, and a full rollout with automatic rollback if service-level objectives degrade. Version prompts and tools, and record the model and tool endpoints used for each turn so incidents can be investigated more reliably. section 8.3.5

How AI Agents in Action, Second Edition covers it

How deep it goes
Details benchmark setup, holistic run checks, repeated testing, and incremental changes, then extends evaluation into automated checks and release gates.
Examples
Tests a fictional knowledge agent with expected and known-bad answers, initially matching expected strings.
What you can do after
Practice test-driven agent development by defining benchmarks, repeating tests, and making minimal prompt or tool changes.
How it connects
Connects evaluation to test-driven development, retrieval-augmented generation, observability, guardrails, and human feedback.

Also explained in

Compare how 15 books explain evaluation →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Agents in Action, Second Edition. The section links point to the parts of the book it draws on.