Chapter 2: Framing and scoping the evaluation
The introduction explains why an AI evaluation needs a shared purpose and scope before a team chooses how to measure it.
After this chapter you will be able to
- Purpose before measurement. Explain why a team should decide what it wants to learn from an AI evaluation before choosing metrics, benchmarks, or toolkits.
- Shared evaluation scope. Describe what framing and scoping make explicit and how those decisions affect later evaluation work.
- Competing stakeholder evaluation aims. Explain how stakeholders in the patient advice chatbot meeting want the evaluation to accomplish different things: establish patient usefulness and engagement, assess the risk of unsafe advice, and provide defensible evidence aligned with medical standards and institutional policy.
- Evaluation goal priorities. Explain how evaluation requests reflect different information needs and decisions, and why evaluators should decide which goals matter now and which can wait.
- Shared capability expectations. Explain how an evaluation can replace conflicting stakeholder assumptions with a shared, realistic picture of what a system reliably handles, where it struggles, and when its performance changes.
- Comparison reference points. Explain that a comparative evaluation asks how a system performs relative to a chosen reference point, such as an earlier version, a competitor, a human baseline, or a performance target.
- Limits and timing of comparison. Distinguish outperforming a reference from being suitable for real-world use, and describe when comparative evaluations are most useful.
- Go/no-go decision criteria. Explain how a go/no-go evaluation determines whether a system should be deployed, continue running, or be taken offline. Describe why the decision needs explicit, context-specific thresholds that account for more than technical performance.
- Limits of average performance. Explain why a system that performs well on average can still pose serious risks, using the medical chatbot example or another failure described here.
- Purpose of risk focused testing. Describe how risk-focused evaluation seeks out likely failures and what it aims to learn from them.
- Evidence of applicable compliance. Explain that compliance evaluation demonstrates, with evidence, whether an AI system meets the regulatory, legal, and organizational requirements that apply to its use. Give an example of a requirement that could be checked.
- Diagnostic evaluation purpose and evidence. Explain when diagnostic evaluation is useful, how its purpose differs from proving system quality, and how teams can use evaluation evidence to locate causes of failures.
- Deliberate evaluation goal priorities. Explain why a team should choose which evaluation goals to pursue first rather than attempting all of them at once.
- Contextual priorities for chatbot evaluation. Apply the medical advice chatbot example to explain which evaluation goals appear most urgent and what further information could change their priority.
- Distinct evaluation objects. Distinguish what is assessed when evaluating the whole medical chatbot, its retrieval module, its generation module, or its external knowledge.
- Goal dependent object selection. Explain how the evaluation goal determines whether to prioritize end-to-end testing, component testing, or both.
- Exact evaluation task. Explain why evaluating an AI system requires identifying the exact task, particularly when a system can perform several tasks.
- Task-dependent evaluation choices. Describe how specifying the task directs evaluation choices and can change the evaluation goal.
- Intended and system task scope. Explain how a scope mismatch arises when stakeholders expect a broader task than the system is designed or able to perform, using the medical chatbot example.
- Distinct tasks with similar responses. Distinguish equivalence mismatch from scope mismatch by explaining why health education and medical triage cannot be evaluated as interchangeable tasks.
- Explicit criteria for success. Explain why an AI task needs explicit criteria for correct, incorrect, acceptable, and unacceptable behavior, especially when responses involve contextual or value-laden judgments.
- Explicit Inputs and Outputs. Describe the inputs and outputs of an AI task concretely, including whether interactions are single-turn or conversational and what form the response should take. Distinguish typical expected inputs from the wider range of allowed inputs the system may encounter.
- Shared Ground Truth Definition. Explain how to establish a shared standard for judging outputs using both criteria for correctness and examples of acceptable and unacceptable responses. State why the standard must resolve fundamental disagreements now, even if detailed rubrics come later.
- Explicit Task Exclusions. Explain how stating what a task does not cover keeps performance claims appropriately scoped and can create checks that the system refrains from out-of-scope behavior.
- Initial Task Risk Assessment. Describe why task definition includes an initial, rough consideration of risk rather than a full risk assessment. Identify the EU AI Act’s four risk categories as an example framework and explain how to choose a relevant framework in a regulated setting.
- Expected System Autonomy. Explain why a task definition must specify how much the AI can act without user involvement and how that affects acceptable failures and evaluation standards. Describe the five-level progression from user-controlled copilot through collaboration and consultation to largely independent and fully autonomous operation.
- Dimensions versus metrics. Distinguish an evaluation dimension—the aspect of a system being assessed for fitness for purpose—from a metric, which specifies how that aspect is measured. Explain why a term such as accuracy can mean either one.
- Decision driven dimension selection. Justify choosing the smallest set of dimensions that informs the current evaluation goal and decision, and describe how that set can change as the system or its risks change.
What it builds on
- What is an AI system (chapter 1) · The idea that an AI system can include multiple components.
Section by section
Introduction
The introduction explains why an AI evaluation needs a shared purpose and scope before a team chooses how to measure it.
- Purpose before measurement: Explain why a team should decide what it wants to learn from an AI evaluation before choosing metrics, benchmarks, or toolkits.
Why it matters: A clear purpose helps the team choose measurements that answer the questions it actually has. - Shared evaluation scope: Describe what framing and scoping make explicit and how those decisions affect later evaluation work.
Why it matters: A shared scope connects the evaluation's purpose to the evidence the team will later collect.
Key terms: evaluation goals, metrics, benchmarks, toolkits, framing and scoping, success, tasks and scenarios, evaluation dimensions
2.1 One room, many expectations
A hospital chatbot scenario shows that stakeholders can agree an AI system needs evaluation while expecting the evaluation to serve different purposes.
- Competing stakeholder evaluation aims: Explain how stakeholders in the patient advice chatbot meeting want the evaluation to accomplish different things: establish patient usefulness and engagement, assess the risk of unsafe advice, and provide defensible evidence aligned with medical standards and institutional policy.
Why it matters: An evaluation plan needs to account for the different questions stakeholders expect it to answer. - Diagnosis versus deployment decision: Distinguish the engineer’s request for an evaluation that helps locate causes of degraded performance from the executive sponsor’s request for a decision about whether the chatbot is safe to deploy.
Why it matters: The answer needed for a deployment decision is different from the information needed to investigate a problem.
Key terms: evaluation plan, adoption, unsafe advice, defensible evidence, diagnostic, performance degrades, single pass–fail answer, safe to deploy
2.2 Defining evaluation goals
This section explains why evaluation goals should be made explicit and prioritized before an evaluation is designed.
- Evaluation goal priorities: Explain how evaluation requests reflect different information needs and decisions, and why evaluators should decide which goals matter now and which can wait.
Why it matters: Clear priorities help keep an evaluation focused on the decisions it needs to support.
Key terms: evaluation goals, information need
2.2.1 Setting expectations
This section explains how evaluation can establish realistic, shared expectations about an AI system’s capabilities and limitations, and when that goal is especially useful.
- Shared capability expectations: Explain how an evaluation can replace conflicting stakeholder assumptions with a shared, realistic picture of what a system reliably handles, where it struggles, and when its performance changes.
Why it matters: A shared view of the system’s strengths and limits helps stakeholders interpret it realistically. - Timing for expectation setting: Describe when setting expectations is especially useful: after a first working prototype, at key milestones, and before presenting the system to external stakeholders.
Why it matters: Checking expectations before a wider demonstration can prevent people from drawing premature conclusions.
Key terms: setting expectations, capabilities and limitations, stakeholders, first working prototype, key milestones, external stakeholders
2.2.2 Comparing
This section explains when to evaluate an AI system by comparing it with a reference point and why outperforming that reference does not establish suitability for use.
- Comparison reference points: Explain that a comparative evaluation asks how a system performs relative to a chosen reference point, such as an earlier version, a competitor, a human baseline, or a performance target.
Why it matters: The reference point determines what a comparison can tell you about the system. - Limits and timing of comparison: Distinguish outperforming a reference from being suitable for real-world use, and describe when comparative evaluations are most useful.
Why it matters: A better score than an alternative should not be mistaken for evidence that a system is ready to use.
Key terms: reference point, human baseline, industry benchmark, performance target, functioning baseline, realistic alternatives, competitive positioning
2.2.3 Deciding go/no-go
This section explains how go/no-go evaluations use context-specific minimum requirements to make deployment or continuation decisions, and when those evaluations are most useful.
- Go/no-go decision criteria: Explain how a go/no-go evaluation determines whether a system should be deployed, continue running, or be taken offline. Describe why the decision needs explicit, context-specific thresholds that account for more than technical performance.
Why it matters: A measured error rate only supports a release decision when you have decided what level of error is acceptable for that use. - Go/no-go decision points: Describe when a go/no-go evaluation is most useful and why it is less necessary during early development.
Why it matters: Matching the evaluation to a real decision avoids imposing release-style gates on exploratory work.
Key terms: go/no-go evaluation, minimum requirements, explicit thresholds, decision criteria, concrete decision points, initial deployment, potential regressions, early development stages
2.2.4 Assessing risk
This section explains why risk assessment goes beyond average performance, how to look for consequential failures, and when that evaluation matters most.
- Limits of average performance: Explain why a system that performs well on average can still pose serious risks, using the medical chatbot example or another failure described here.
Why it matters: A strong average result does not show whether the system is safe for every situation or user it may encounter. - Purpose of risk focused testing: Describe how risk-focused evaluation seeks out likely failures and what it aims to learn from them.
Why it matters: Finding consequential failures early leaves an opportunity to mitigate them. - Priority and timing of assessment: Distinguish when risk assessment is especially important from when it is less useful.
Why it matters: The value of a risk assessment depends on both the potential for harm and whether the system is mature enough to assess.
Key terms: assess risk, accuracy, effectiveness, risk-focused evaluation, vulnerabilities, high-risk scenarios, safety- and trust-critical contexts, before deployment, external audits
2.2.5 Ensuring compliance
This section explains how compliance evaluation produces evidence that an AI system meets applicable requirements, how that evidence is checked, and why it matters for trust.
- Evidence of applicable compliance: Explain that compliance evaluation demonstrates, with evidence, whether an AI system meets the regulatory, legal, and organizational requirements that apply to its use. Give an example of a requirement that could be checked.
Why it matters: The requirements that apply to a system determine what its compliance evaluation must substantiate. - Repeatable compliance checks: Describe how explicit requirements can be turned into repeatable, documented tests and controls, and identify when formal compliance checks may take place.
Why it matters: Repeatable, documented checks make compliance claims reviewable at deployment and when circumstances change. - Trust beyond box checking: Justify why documented compliance evidence matters beyond avoiding fines or legal exposure.
Why it matters: Compliance evidence can build confidence in the system and the organization, not just reduce legal exposure.
Key terms: compliance evaluation, regulatory, legal, and organizational requirements, evidence, test cases, controls, verification procedures, formal audits, formal checkpoints, trust, accountability
2.2.6 Diagnosing and debugging
This section explains how evaluation helps diagnose AI system failures and why academic benchmark results alone may not establish real-world readiness.
- Diagnostic evaluation purpose and evidence: Explain when diagnostic evaluation is useful, how its purpose differs from proving system quality, and how teams can use evaluation evidence to locate causes of failures.
Why it matters: Finding where a failure comes from helps a team choose a relevant fix instead of merely reporting a poor score. - Academic benchmarks and deployment readiness: Distinguish the common purpose of academic AI evaluations from determining whether a system is ready for real-world use.
Why it matters: Benchmark results can be useful evidence without answering whether a system is ready for an applied setting.
Key terms: diagnosis, root causes, error distributions, data segments, logs, benchmark datasets, controlled experimental conditions, baselines, real-world use
2.2.7 Negotiating and prioritizing evaluation goals
This section explains why evaluation goals must be prioritized and shows how stakeholder concerns and deployment context shape those priorities.
- Deliberate evaluation goal priorities: Explain why a team should choose which evaluation goals to pursue first rather than attempting all of them at once.
Why it matters: Prioritizing goals helps the evaluation provide useful evidence for a real decision. - Contextual priorities for chatbot evaluation: Apply the medical advice chatbot example to explain which evaluation goals appear most urgent and what further information could change their priority.
Why it matters: The right priorities depend on the decision at hand and the risks and constraints surrounding it.
Key terms: evaluation goals, prioritizing evaluation goals, risk assessment, go/no-go evaluation, compliance, diagnostic evaluation, deployment readiness
2.3 Determining the objects of the evaluation
The section uses a medical RAG chatbot to show that evaluation objects can range from the whole system to individual components, and that the right choice depends on the evaluation goal.
- RAG information flow: Trace how the medical RAG chatbot uses an external knowledge source and a language model to answer a question.
Why it matters: The information flow makes clear why retrieval, generation, and the knowledge source can be evaluated separately. - Distinct evaluation objects: Distinguish what is assessed when evaluating the whole medical chatbot, its retrieval module, its generation module, or its external knowledge.
Why it matters: A good overall answer can hide component problems, while a component test alone cannot establish how the chatbot behaves for users. - Goal dependent object selection: Explain how the evaluation goal determines whether to prioritize end-to-end testing, component testing, or both.
Why it matters: Choosing the wrong evaluation object can produce results that do not answer the question you set out to investigate.
Key terms: retrieval-augmented generation (RAG), external knowledge source, language model, retrieved context, end-to-end-system, retrieval module, generation module, external knowledge, evaluation goal, object of evaluation
2.4 Defining the task to be evaluated
This section explains why an evaluation must specify the AI system’s exact task before deciding how to assess it.
- Exact evaluation task: Explain why evaluating an AI system requires identifying the exact task, particularly when a system can perform several tasks.
Why it matters: Naming the system is not enough to establish what its evaluation should test. - Task-dependent evaluation choices: Describe how specifying the task directs evaluation choices and can change the evaluation goal.
Why it matters: Different tasks call for different tests, and some reveal safety concerns that a performance-focused evaluation could miss.
Key terms: task definition, exact task, ill-defined, evaluation dimensions, metrics, risk assessment
2.4.1 Problems with AI task definitions
This section explains three ways unclear AI task definitions can undermine an evaluation: scope mismatch, equivalence mismatch, and vague criteria for success.
- Intended and system task scope: Explain how a scope mismatch arises when stakeholders expect a broader task than the system is designed or able to perform, using the medical chatbot example.
Why it matters: A scope mismatch can make an evaluation test capabilities the system was never meant to provide. - Distinct tasks with similar responses: Distinguish equivalence mismatch from scope mismatch by explaining why health education and medical triage cannot be evaluated as interchangeable tasks.
Why it matters: Keeping similar-looking tasks distinct prevents misleading judgments about performance and safety. - Explicit criteria for success: Explain why an AI task needs explicit criteria for correct, incorrect, acceptable, and unacceptable behavior, especially when responses involve contextual or value-laden judgments.
Why it matters: Clear success criteria give evaluators a shared basis for judging responses.
Key terms: intended task, system task, scope mismatch, equivalence mismatch, health education, medical triage, criteria of correctness and success, correct, incorrect, acceptable
2.4.2 Defining an AI task well
This section turns a vague AI task into an evaluable definition by clarifying its inputs, ground truth, boundaries, risk, and expected autonomy.
- Explicit Inputs and Outputs: Describe the inputs and outputs of an AI task concretely, including whether interactions are single-turn or conversational and what form the response should take. Distinguish typical expected inputs from the wider range of allowed inputs the system may encounter.
Why it matters: A system can look good on tidy test questions yet fail on the messages real users send. - Shared Ground Truth Definition: Explain how to establish a shared standard for judging outputs using both criteria for correctness and examples of acceptable and unacceptable responses. State why the standard must resolve fundamental disagreements now, even if detailed rubrics come later.
Why it matters: Without a shared standard, evaluators can produce inconsistent judgments that cannot be reconciled later. - Explicit Task Exclusions: Explain how stating what a task does not cover keeps performance claims appropriately scoped and can create checks that the system refrains from out-of-scope behavior.
Why it matters: A medical-information chatbot should neither be marked down for not diagnosing nor allowed to diagnose when pushed. - Initial Task Risk Assessment: Describe why task definition includes an initial, rough consideration of risk rather than a full risk assessment. Identify the EU AI Act’s four risk categories as an example framework and explain how to choose a relevant framework in a regulated setting.
Why it matters: Even a preliminary risk check helps keep the task’s regulatory and societal implications in view. - Expected System Autonomy: Explain why a task definition must specify how much the AI can act without user involvement and how that affects acceptable failures and evaluation standards. Describe the five-level progression from user-controlled copilot through collaboration and consultation to largely independent and fully autonomous operation.
Why it matters: The same failure can demand a different evaluation response depending on whether a human can review or stop the system’s actions. - Decomposing Bundled Tasks: Describe when an apparently hard-to-define task should be split into smaller tasks that can be evaluated independently.
Why it matters: Separating bundled tasks can make each definition of success clear enough to evaluate.
Key terms: inputs, outputs, allowed input, expected input, ground truth, intensional definition, extensional definition, out of scope, task exclusions, risk level
2.5 Choosing evaluation dimensions
This section explains what evaluation dimensions are, how they differ from metrics, and how to choose a focused set for a particular evaluation decision.
- Dimensions versus metrics: Distinguish an evaluation dimension—the aspect of a system being assessed for fitness for purpose—from a metric, which specifies how that aspect is measured. Explain why a term such as accuracy can mean either one.
Why it matters: Separating what you care about from how you measure it makes the evaluation plan clearer. - Overlapping evaluation dimensions: Explain why evaluation dimensions are rarely assessed in isolation, using a connection between two dimensions from the section.
Why it matters: Connections between dimensions can reveal consequences that a narrow, isolated assessment would miss. - Decision driven dimension selection: Justify choosing the smallest set of dimensions that informs the current evaluation goal and decision, and describe how that set can change as the system or its risks change.
Why it matters: A focused scope makes the evaluation useful for the decision at hand without trying to assess everything at once.
Key terms: evaluation dimensions, metrics, safety, accuracy, robustness, fairness, explainability, efficiency, societal impact, evaluation scope
2.6 References
This section lists references cited in the chapter on framing and scoping AI evaluation.
2.7 Summary
Where this is used later
- “How do we measure this?” (chapter 3) · Addresses measurement after the evaluation has been framed.
- Managing and maintaining metrics (chapter 3) · the go/no-go evaluation goal
- What is a metric (chapter 3) · The task to be evaluated.
- (chapter intro) (chapter 4) · The task being evaluated helps determine which situations are relevant.
- Fitness to the evaluation task (chapter 4) · The evaluation task definition provides the basis for judging dataset fit.
- Accuracy and correctness (chapter 4) · The task definition against which facts, labels, and judgments are assessed.
- (chapter intro) (chapter 3) · The aspects of performance selected for evaluation.
- Construct validity (chapter 3) · Assumes an evaluation dimension or quality has been chosen to measure.
Concepts explained in this chapter
- ground truthGround truth is the reference standard used to judge whether system outputs are correct, acceptable, or incorrect.
- evaluation goalThe intended purpose or decision that an evaluation is meant to support.
- evaluation criterionA stated condition or standard used to determine whether a system satisfies a decision requirement.
- evaluation taskAn evaluation task is an activity an AI system is expected to perform and whose outcomes can be assessed.
- task definitionA task definition specifies the activity an AI system is expected to perform and the conditions that define its scope and success.
- metricA quantitative or scored measure used to assess a system against an evaluation criterion.
- evaluation activityA step in the process of planning, executing, interpreting, or acting on an AI evaluation.
- evaluation objectThe AI system, subsystem, or component whose performance or properties are being assessed.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
