A rubric tells an LLM judge what “good” means for a task. Written in plain language as the judge’s system prompt, it sets the standard behind the evaluation signal. section 2.2.3 section 2.2
Comparing research answers
A pairwise judge can prioritize factual accuracy against a reference, grounding in retrieved sources, and clarity. It returns a winner—LEFT, RIGHT, or TIE—and brief feedback. section 2.2.3
Improving a support answer
For a customer’s yes-or-no question, a one-sentence rubric asks how directly the answer addresses it. The judge can use that signal to prefer a prompt that puts the verdict first. section 1.2.1
Figure 2.5 details the internal process of how LLM-as-Judge uses pairwise evaluation to compare multiple outputs against the same rubric without having position bias modify the answer.

Trace how the same rubric is applied to both answer orders, filtering out verdicts that change with position. section 2.2.3






