liveBook · Manning

Calibration data

As explained in LLM Evaluation and Alignment, The Foundational Ideas

A calibration set is a collection of responses spanning a quality range with human ratings used to map raw model scores to calibrated values.

Calibration data is a human-rated response set used to check whether an AI judge's scores or preferences reflect intended quality judgments. It should match the task or production prompt distribution and cover meaningful quality variation. section 4.4.4 section 8.3.2

Build and use calibration data through these steps:

  • Sample responses from the actual evaluation or production distribution; larger preference-calibration exercises may use 500–1,000 prompts with two responses per prompt. section 8.3.2
  • Collect ratings or pairwise rankings from two or three calibrated human labelers, then measure human–human agreement as the comparison ceiling. section 8.3.2
  • Run the AI judge with the same prompt, constitution, and reasoning structure intended for production, then compare its agreement with human consensus. section 8.3.2
  • For numeric judges, map raw scores to human ratings; for unreliable absolute scores, consider percentile ranking or a human-validated pass/fail threshold. section 4.4.4

Restoring a compressed score range

Suppose a judge assigns scores from 3.5 to 4.8 while humans rate the same responses from 1 to 5. A calibration set spanning the full quality range can define a mapping that spreads the judge's outputs across the human scale. Recalibrate when monitoring detects drift. section 4.4.4

How LLM Evaluation and Alignment, The Foundational Ideas covers it

How deep it goes
Defines sampling, human labeling, agreement comparison, and score mapping, while deferring detailed statistical calibration methods.
Examples
Uses full-range human ratings, production-distribution prompt samples, paired responses, and compressed-score normalization.
What it assumes
Assumes human rating or ranking procedures and basic agreement measurement, while treating calibration as an explicit validation step.
What you can do after
Diagnose judge miscalibration, normalize scores, compare AI–human agreement, or select ranking and pass/fail fallbacks.
How it connects
Connects to AI judges, human agreement, score normalization, rank-based transformation, binary decisions, and drift detection.

Also explained in

Compare how 6 books explain calibration data →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.