A chatbot can give a fluent answer that is unhelpful or unfaithful to the available facts. Human-in-the-loop evaluation uses people’s judgments as a quality signal and keeps human oversight involved when automated scores are unreliable or decisions are consequential. section 1.1.2 section 4.4.4
Human judgment is especially useful for open-ended tasks with multiple valid responses, such as dialogue or summarization. Unlike a programmatically verifiable answer, quality may depend on context, tone, faithfulness, and policy. Optimizing a fixed metric alone can encourage shortcuts that raise the score without improving the response. section 1.1.2 section 7.1.1
Evaluating a customer service response
A customer asks where an order is after waiting three weeks. Rather than compare the chatbot’s reply with one fixed answer, an evaluator can check whether it is polite, relevant, coherent, and faithful to the order data. section 1.1.2
To use human judgments reliably alongside an LLM judge:
- Give annotators task context, scoring instructions, examples, and a rubric defining each score level. Refine unclear instructions and run pilot studies. section 4.2.1
- Maintain a fixed golden dataset with human judgments across the quality spectrum, including ambiguous edge cases. Re-run the judge on it weekly or after suspected model updates. section 4.4.4
- Route a random sample of evaluations to humans for auditing; also review low-confidence cases or scores near decision boundaries. section 4.4.4
- Track agreement over time: use Cohen’s kappa for categorical decisions and correlation for scalar scores. Define a threshold for alerting the team and pausing automated decisions. section 4.4.4
Figure 4.2 Detailed instructions (top) and examples (bottom) provided to human labelers at Amazon Mechanical Turk. This approach of providing both explicit guidelines and illustrative examples is essential for high-quality human annotation and is a best practice when prompting LLM-as-a-judge. Source: Lee et al. (2024)
The example annotation instructions show how explicit guidelines and examples support consistent judgments. section 4.2.1
Human judgments can also train learned evaluation metrics, such as COMET. During reward-model training, monitor scores against a held-out human preference signal: rising reward scores alongside flat or falling human ratings can indicate reward hacking. section 3.5 section 7.6.1
Human review has cost and scaling limits, and unclear instructions can produce inconsistent ratings. For high-stakes decisions, retain human safeguards even when hallucination detection is in place: detection cannot eliminate every error. section 4.2.1 section 5.4.5



























