In LLM-as-a-judge evaluation, chain-of-thought means asking a judge to work through rubric criteria explicitly before returning a score. The resulting reasoning makes the basis of a verdict inspectable; a bare number does not. section 10.3.1
A reasoning-first scoring prompt has practical trade-offs:
- Require criterion-level reasoning before the score so the judge engages with the evidence named by the rubric. section 10.3.1
- Inspect the reasoning to diagnose whether a wrong score came from misunderstanding the task, missing context, or applying the rubric differently. section 10.3.1
- Budget for extra tokens and latency; a fluent explanation does not prove the judgment is correct. section 10.3.1
Bare score versus reasoning first
A prompt such as “rate this response 1–5” allows a surface-impression verdict. Asking for a short justification for each rubric dimension before the score leaves reasoning to inspect when the score looks wrong. section 10.3.1




















