A gold standard is an expert-produced reference answer or output treated as the target for evaluating an AI-generated candidate. Reference-based evaluation compares the candidate with one or more such outputs, while reference-free evaluation judges quality without them. section 1.1 section 2.1.1
Use gold standards when:
- A task has a trusted expert answer, such as a translation, summary, or other target output. section 1.1
- Several valid answers exist, making exact matching insufficient; multiple references can represent acceptable variation. section 2.1.1
- You need automated signals such as word overlap or semantic similarity, while recognizing that these remain proxies for quality. section 2.1.1 section 7.1.1
Evaluating a translation
Given a Chinese sentence, compare two machine translations with three professional human translations. The better candidate shares more appropriate words and phrases with the references, though overlap alone does not fully measure meaning or quality. section 2.1.1
Figure 1.1 A taxonomy of AI evaluation methods, distinguishing between verifiable tasks and open-ended tasks.
Locate gold standards on the reference-based branch for open-ended tasks; verifiable tasks instead use objectively checkable answers. section 1.1




























