liveBook · Manning

Single-judge debate

As explained in AI Model Evaluation

A debate-style evaluation technique in which one model invocation argues for both alternatives and then judges the arguments.

Single-judge debate evaluates two responses by making one model invocation argue the strongest case for each alternative before weighing the arguments and issuing a verdict. section 10.3.3

Use a staged prompt that separates advocacy from judgment:

  • Provide the user query, both responses, and criteria such as accuracy, relevance, and clarity. section 10.3.3
  • Build a case for Response A, build a case for Response B, weigh both arguments, then render a verdict. section 10.3.3
  • Require structured JSON containing both cases, the weighing, and a winner of A, B, or tie with justification. section 10.3.3

Evaluation flow

An impartial evaluator first argues why Response A better meets the criteria, then why Response B does, compares the strength of those arguments, and finally returns the supported verdict. section 10.3.3

How AI Model Evaluation covers it

How deep it goes
Specifies prompt stages and JSON output while remaining one model’s single reasoning trace.
Examples
Uses two responses, evaluation criteria, advocate cases, argument weighing, and a winner-or-tie JSON verdict.
What it assumes
Assumes two candidate responses and explicit comparison criteria.
What you can do after
Apply it via a staged prompt that produces comparable arguments and a structured verdict.
How it connects
Connects LLM-as-a-judge, pairwise comparison, and reasoning-first prompting.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Model Evaluation. The section links point to the parts of the book it draws on.