liveBook · Manning

Abstention

As explained in AI Model Evaluation

Abstention is an evaluation behavior in which a judge declines to issue a verdict when the available information is insufficient for a meaningful judgment.

Abstention is an evaluation behavior that lets a judge decline to score when the available context cannot support a meaningful verdict. It prevents missing evidence from becoming a confident-looking but ungrounded judgment. Unlike escalation, which asks an uncertain judge to try harder, abstention recognizes that more reasoning or model strength cannot compensate for absent information. section 10.5 section 10.5.3

Implement abstention as an explicit path in the judging contract and the surrounding evaluation pipeline:

  • Require the judge to score only when the supplied context is sufficient to verify the response. section 10.5.3
  • Return a distinct status such as scored or abstained, with a null score and a reason identifying what information is missing. section 10.5.3
  • Route abstained cases to a stronger judge with better context or to human review, while reporting abstention separately from escalation. section 10.5.3
  • Monitor the defer rate as a possible data-pipeline signal, especially when it rises over time. section 10.7

Fact-checking with incomplete source context

A judge receives a query, response, and source context. If the context verifies every relevant claim, it returns {"status":"scored","score":1-5,"reason":"..."}. If a claim cannot be checked, it returns {"status":"abstained","score":null,"reason":"missing information needed to verify the claim"} and routes the case onward. section 10.5.3

How AI Model Evaluation covers it

How deep it goes
Defines the behavior, output contract, routing, and reporting distinction, but does not specify a particular model or implementation library.
Examples
Uses factual-support judging with query, response, source context, scored-or-abstained status, nullable score, and missing-evidence reason.
What it assumes
Assumes familiarity with LLM judges, evaluation pipelines, source context, escalation, and structured scoring outputs.
What you can do after
Operate an evaluation pipeline that exposes missing grounding, routes deferred cases, and separates abstention rates from escalation rates.
How it connects
Connects abstention to escalation, tiered judging, evaluation pipelines, stronger judges, and human review.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Model Evaluation. The section links point to the parts of the book it draws on.