liveBook · Manning

Model calibration

As explained in AI Model Evaluation

Calibration is the agreement between a model's predicted confidence and the observed frequency of its outcomes.

Model calibration is the agreement between predicted confidence and observed outcomes. For an LLM judge, a confidence label is useful for routing decisions only if it predicts correctness: high-confidence verdicts should be correct more often than lower-confidence ones, and the stated confidence should reflect observed accuracy. Accuracy, consistency, and confidence calibration are distinct properties. section 10.5.1

Check the signal before scaling a tiered judging system. Run the first- and second-tier judges on the same offline set, then measure first-tier accuracy by its self-reported confidence level. If high-confidence verdicts are wrong nearly as often as medium-confidence verdicts, adjusting routing thresholds will not make the confidence label a useful signal. section 10.5.1

Calibration can also refer to anchoring an LLM judge’s scores with labeled examples. This few-shot calibration teaches the judge what scores such as 5 and 2 look like under a particular rubric. It can improve alignment with human standards, but it is not the same as measuring whether confidence predicts correctness. section 9.6.1 section 10.5.1

Before running an LLM judge at scale:

  • Compare judgments with human-labeled gold examples, and use representative scored cases to anchor the rubric. section 9.10.3 section 9.6.1
  • Repeat examples to spot variance, then revise the prompt and check consistency across random seeds or few-shot contexts. section 9.10.3
  • Validate reliability separately against human ratings, gold labels, known examples, or canary sets. Anchoring scores alone is not full validation. section 9.11
  • Log calibration or human-alignment results alongside the prompt and model versions, rubric, sampling strategy, and dataset snapshot so the run can be traced and reproduced. section 9.10.2

Calibrating a movie recommendation judge

A judge scores recommendations on criteria such as diversity and discovery. Give it a few recommendations pre-scored by humans or domain experts to illustrate the expected reasoning and score distribution. Then compare prompt versions using stability across reruns or correlation with human ratings; do not treat the resulting score as objective truth without validation. section 9.6.1 section 9.11

How AI Model Evaluation covers it

How deep it goes
Provides gold-example, rerun, prompt-iteration, and confidence-stratification checks, while distinguishing calibration from full validation.
Examples
Uses a movie recommendation judge with human-scored examples and an offline comparison of first- and second-tier judges.
What you can do after
Evaluate and understand whether judge scores align with gold examples and whether confidence levels predict correctness.
How it connects
Connects calibration to few-shot prompting, human ratings, evaluation rubrics, and tiered judging.

Also explained in

Compare how 7 books explain model calibration →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Model Evaluation. The section links point to the parts of the book it draws on.