Requirement compliance measures whether an answer follows the request’s format, required criteria, scope, length, and language. Treat it as a binary rubric item: PASS when the response satisfies the requested constraints; FAIL when it breaks the format, omits required criteria, or exceeds the requested scope. section 10.2.4
Evaluate requirement compliance against explicit constraints:
- Requested format: Check whether the answer uses the required structure, such as a table or bullets. section 10.2.4
- Required criteria: Check whether every requested comparison point or condition appears. section 10.2.4
- Scope, length, and language: Fail responses that go beyond the requested scope or violate specified length or language constraints. section 10.2.4
Use requirement compliance alongside answer relevancy and citation reliability. A requirement-compliance pass rate tracks the fraction of evaluated answers that pass this rubric item; an overall pass rate counts cases passing all three items. section 10.2.4
Rubric-based evaluation
Provide the judge with the user query, final answer, rubric, evidence pack, and optional reference answer. Require one JSON object containing PASS or FAIL for each rubric item, including requirement compliance. Judge only the supplied evidence and criteria, without rewarding verbosity, style, or friendliness. section 10.3.2
