Labeled data pairs examples with annotations that represent target outputs, judgments, categories, or other reference outcomes. In offline evaluation, predictions are compared with these labels to compute metrics, assess performance, and uncover issues. section 2.2.1
Use labeled data to:
- Measure classification accuracy against ground-truth categories. section 2.2.1
- Calibrate an LLM judge by comparing its judgments with known human-labeled examples. section 9.10.3
- Validate evaluation behavior before scaling, while checking prompt ambiguity, repeated-run variance, and consistency across contexts. section 9.10.3
Movie recommendation evaluation
A classification dataset can pair each example with a ground-truth category and support accuracy measurement. For ranking, historical movie clicks, impressions, watches, saves, or ratings can provide comparison signals, but a click may reflect prominent placement rather than genuine preference; interpret resulting metrics accordingly. section 2.2.1







