Evaluation measures whether a model, agent, or AI system performs a task accurately and reliably, including on edge cases. It reveals what the system does; alignment changes its behavior toward the intended goal. A convincing demo is not enough: a system can produce fluent answers while giving incorrect information. section 1.0 section 1.1
Choose an evaluation method based on the task. Verifiable tasks, such as many coding and math problems, have objectively checkable outputs. Open-ended tasks, such as report generation or creative writing, may have many valid answers and require judgment about qualities such as helpfulness and tone. section 1.5 section 1.0
Figure 1.1 A taxonomy of AI evaluation methods, distinguishing between verifiable tasks and open-ended tasks.
Use this taxonomy to navigate from task type to evaluation approach. section 1.1
For open-ended outputs, select evidence and metrics that fit the desired quality:
- Reference-based evaluation compares an output with an expert-produced answer. Word-overlap metrics are inexpensive checks, but may miss correct paraphrases. section 1.5 section 2.0
- Reference-free evaluation assesses properties of the output without a gold-standard answer, sometimes using expert criteria. section 1.5
- Semantic metrics assess similarity of meaning, while an LLM judge can approximate human judgment at scale. Neither makes judgment automatic or infallible. section 1.1 section 4.5
Use evaluation as a repeating engineering loop:
- 1. Define success criteria and metrics for the actual task. section 1.3.1
- 2. Test the system and measure each criterion. section 1.3.1
- 3. Analyze failures to identify their causes, rather than relying on an aggregate score. section 1.3.1
- 4. Change the system to address the failures, then evaluate again. section 1.3.1
Customer service chatbot
In 500 test conversations, a chatbot achieves 87% factual accuracy. Error analysis finds that it invents shipping dates when the database returns null. The team changes the prompt to handle missing information and adds retrieval of shipping policies. Re-evaluation shows 94% accuracy, but further errors still need investigation. section 1.3.1
Evaluate at the right level. For a retrieval-augmented agent, a final answer score may reveal a failure without locating it. Inspect the execution trace: the information need, search query, tool choice, retrieved results, stop decision, and final synthesis can each fail independently. Assess retrieval quality before interpreting generation metrics. section 6.4.1 section 6.5
Offline evaluation supports development and testing, typically in batches with available or constructible reference data. Online evaluation monitors production interactions in parallel with serving; it records scores without blocking responses. Reference answers and timely human feedback are often unavailable online, and judge calls add cost. section 4.4.1
Table 4.6 Comparing online and offline evaluation contexts
|
Dimension
|
Online Evaluation
|
Offline Evaluation
|
|
Purpose
|
Production monitoring
|
Development and testing
|
|
Execution
|
Parallel to serving, non-blocking
|
Batch processing
|
|
Primary Constraints
|
Cost and data availability
|
Accuracy and comprehensiveness
|
|
Reference Data
|
Usually unavailable
|
Available, or constructible
|
|
Typical scale
|
Every request or a sampled subset
|
Hundreds to thousands
|
|
Model choice
|
Small, fast, lower cost models
|
Frontier models for maximum accuracy
|
Compare the data, cost, and accuracy constraints of online and offline evaluation. section 4.4.1
Interpret scores in context. LLM judges can favor response position, verbosity, or their own model family; calibrate and monitor them with regression tests and human verification. Do not make a judge the sole arbiter of safety-critical decisions. When a specification defines expected behavior, turn its clauses and discovered failures into repeatable tests. section 4.5 section 8.4.4




























