liveBook · Manning

Debate-style judging

As explained in AI Model Evaluation

An evaluation pattern in which competing cases are constructed for alternatives before an evaluator weighs them and renders a judgment.

Debate-style judging makes an evaluator construct the strongest case for each alternative before weighing those cases and issuing a verdict. This delays premature preference and reduces the risk of rationalizing an initial impression. section 10.3.3

A pairwise judging flow uses four explicit stages:

  • Argue for Response A using specific evidence and the stated criteria. section 10.3.3
  • Argue for Response B with the same requirement for evidence. section 10.3.3
  • Weigh both arguments, testing which case survives scrutiny and which relies on weaker or unsupported claims. section 10.3.3
  • Render a verdict only after both cases have been constructed. section 10.3.3

Self-debate for pairwise comparison

One model can play both advocate roles within a single invocation, comparing two responses against criteria such as accuracy, relevance, and clarity. The process restructures one reasoning trace rather than creating independent judges. section 10.3.3

How AI Model Evaluation covers it

How deep it goes
Specifies advocacy, counter-advocacy, weighing, and verdict stages, while remaining one model’s single reasoning trace.
Examples
Uses two candidate responses evaluated for accuracy, relevance, and clarity through a self-debate prompt.
What it assumes
Assumes two responses, a user query, and explicit decision criteria supplied to an impartial evaluator.
What you can do after
Evaluate two responses by constructing evidence-based cases for each and rendering a delayed comparative verdict.
How it connects
Connects self-critique, reasoning-first prompting, adversarial judging, and pairwise comparison.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Model Evaluation. The section links point to the parts of the book it draws on.