liveBook · Manning

Tier 2

As explained in AI Model Evaluation

An evaluation tier is a level in a multi-stage evaluation system with a defined evaluator cost, capability, and escalation position.

Tier 2 is the expensive, low-volume evaluator in a tiered judging system. A fast Tier 1 judge handles every example, while Tier 2 re-evaluates cases that the first pass cannot resolve confidently. section 10.5.1

Route an example upward when the first evaluator signals uncertainty through:

A limitation of confidence-based routing

A Tier 1 judge may confidently accept a fabricated figure when fluent surrounding prose makes the answer appear credible. Because the verdict is neither borderline nor low-confidence, it never reaches Tier 2. Tiering therefore concentrates stronger evaluation on visible uncertainty but can miss confidently consistent errors. section 10.5.2

How AI Model Evaluation covers it

How deep it goes
Defines routing signals and validation needs, but stops short of eliminating confidently wrong Tier 1 judgments.
Examples
Uses borderline scores, unstable reruns, swapped-order tests, low-confidence outputs, and confidently accepted fabricated figures.
What it assumes
Assumes a Tier 1 evaluator, decision thresholds, repeated evaluations, and an offline set shared with Tier 2.
What you can do after
Configure escalation rules and validate whether Tier 1 confidence predicts correctness before relying on routing.
How it connects
Connects LLM-as-a-judge with escalation, confidence calibration, repeated sampling, and human review.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Model Evaluation. The section links point to the parts of the book it draws on.