Accuracy measures how often an LLM judge’s verdicts agree with known-correct labels in a trusted reference dataset. section 10.4.1
Accuracy and stability measure different things: a judge may be right often but vary across reruns, or repeat the same incorrect verdict. section 10.4.1
Correctness is not consistency
A judge can reach correct verdicts most of the time yet give different outputs for the same input on different runs. A judge that repeats its verdict reliably may still always be wrong. section 10.4.1
















































