Alignment is the broader goal of making an LLM judge’s behavior, reasoning, and outputs match evaluation objectives and human judgment. Calibration supports this goal by anchoring scores to labeled gold-standard examples. section 9.6.1
Align an evaluation setup through several complementary practices:
- Design clear prompts and specify the evaluation rubric. section 9.6.1
- Select models trained with objectives appropriate to the evaluation task. section 9.6.1
- Validate judgments continuously against human ratings or historical baselines. section 9.6.1
Movie recommender judge
A movie recommendation evaluator receives a structured prompt and a few human-scored examples showing what scores such as 5 and 2 represent. These gold examples reduce arbitrary scoring, limit subjective drift, and make the evaluator’s reasoning more traceable. section 9.6.1





