liveBook · Manning

Absolute judgment

Absolute judgment is an evaluation method in which an output is assessed independently, often by assigning it a score, rather than comparing it with another output.

How 2 Manning books explain it

LLM Evaluation and Alignment, The Foundational Ideas

An absolute judgment assigns a standalone quality or acceptability score to one response without comparing it with another response.

How it is used
Uses standalone scoring to evaluate one response against explicit quality criteria.
How deep it goes
Defines the contrast with pairwise judgment and stresses clearer rubrics, while deferring detailed scoring mechanics.
Examples
Uses 1–5 coherence ratings for news summaries and structured reasoning steps as concrete evaluation artifacts.
What you can do after
Apply G-Eval-style scoring by defining criteria, generating evaluation steps, and producing a final assessment.

Read how LLM Evaluation and Alignment, The Foundational Ideas explains absolute judgment →

Evaluating AI Systems

A method of assessment in which a judge evaluates an output independently, such as by assigning it a score on a scale.

How it is used
Uses single-output judgments against fixed standards to enforce thresholds such as safety, compliance, or deployment readiness.
How deep it goes
Defines absolute versus relative and binary versus scalar judgments, then outlines their trade-offs without giving a detailed scoring procedure.
Examples
Uses supported claims, tumor detection, chatbot safety, and 1–5 clarity ratings to illustrate judgment formats.
What you can do after
Evaluate whether a single output meets a fixed standard, including a minimum safety or compliance threshold.

Read how Evaluating AI Systems explains absolute judgment →

Related concepts

Questions about absolute judgment

What is absolute judgment?

Absolute judgment is an evaluation method in which an output is assessed independently, often by assigning it a score, rather than comparing it with another output.

Which Manning books explain absolute judgment?

LLM Evaluation and Alignment, The Foundational Ideas and Evaluating AI Systems.

What is the difference between absolute judgment and relative judgment?

Absolute judgment is an evaluation method in which an output is assessed independently, often by assigning it a score, rather than comparing it with another output. A method of assessment in which judges compare outputs with one another rather than assigning each an independent score.

What is absolute judgment a kind of?

Absolute judgment is a kind of judgment format. A form in which a judgment is expressed, such as a binary choice, a scalar value, or a comparison between outputs.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.