An evaluation task is an activity an AI system is expected to perform and whose outcomes can be assessed. Defining the task precisely helps determine what to test, what success means, and which risks matter. A system may perform one task or several, including tasks its designers did not anticipate. section 2.4 section 2.7
Task definition is distinct from choosing the system component to evaluate. For a legal chatbot, testing the language model’s factual answers may miss how the full system handles ambiguity, presents answers, or refers users to a lawyer. The relevant task may involve navigating procedures or de-escalating conflict, not just answering legal questions. section 1.5
Define the task before choosing metrics or data:
- Specify expected inputs and outputs, acceptable behavior, boundaries, and potential risks. section 2.7
- Identify the users, scenarios, and decisions the evaluation must support. These choices shape the dimensions, metrics, data, and methods. section 2.0
- Check whether the task changes the evaluation goal: a system intended to advise users may need risk assessment as well as performance testing. section 2.4
Medical information versus triage
A medical chatbot providing general educational information may be assessed for factual accuracy and clarity. If it advises patients on next steps, safety, escalation, and appropriate disclaimers become central. A dataset of short factual questions may suit the first task but miss users seeking guidance tailored to their circumstances. section 2.4 section 4.5.1
A dataset can appear relevant while assuming a different user intent, operational constraint, or standard of good behavior. For example, a general chatbot dataset that rewards long answers may be unsuitable for clinical triage, where brevity, warnings, and escalation matter. Compare the dataset’s documented task and annotation guidelines with the intended task before reusing it. section 4.5.1
