Absolute judgment
Absolute judgment is an evaluation method in which an output is assessed independently, often by assigning it a score, rather than comparing it with another output.
LLM Evaluation and Alignment, The Foundational Ideas
An absolute judgment assigns a standalone quality or acceptability score to one response without comparing it with another response.
- How it is used
- Uses standalone scoring to evaluate one response against explicit quality criteria.
- How deep it goes
- Defines the contrast with pairwise judgment and stresses clearer rubrics, while deferring detailed scoring mechanics.
- Examples
- Uses 1–5 coherence ratings for news summaries and structured reasoning steps as concrete evaluation artifacts.
- What you can do after
- Apply G-Eval-style scoring by defining criteria, generating evaluation steps, and producing a final assessment.
Read how LLM Evaluation and Alignment, The Foundational Ideas explains absolute judgment →
Evaluating AI Systems
A method of assessment in which a judge evaluates an output independently, such as by assigning it a score on a scale.
- How it is used
- Uses single-output judgments against fixed standards to enforce thresholds such as safety, compliance, or deployment readiness.
- How deep it goes
- Defines absolute versus relative and binary versus scalar judgments, then outlines their trade-offs without giving a detailed scoring procedure.
- Examples
- Uses supported claims, tumor detection, chatbot safety, and 1–5 clarity ratings to illustrate judgment formats.
- What you can do after
- Evaluate whether a single output meets a fixed standard, including a minimum safety or compliance threshold.
Related concepts
- relative judgmentcontrasts with · A method of assessment in which judges compare outputs with one another rather than assigning each an independent score.
- judgment formatkind of · A form in which a judgment is expressed, such as a binary choice, a scalar value, or a comparison between outputs.
- judgmentkind of · Judgment is the process or capacity to assess information and circumstances, interpret them in context, and form an evaluation or decision.
- thresholduses · A threshold is a boundary value used to determine whether a measured quantity meets a criterion or changes a classification, decision, or action.
Questions about absolute judgment
What is absolute judgment?
Absolute judgment is an evaluation method in which an output is assessed independently, often by assigning it a score, rather than comparing it with another output.
Which Manning books explain absolute judgment?
LLM Evaluation and Alignment, The Foundational Ideas and Evaluating AI Systems.
What is the difference between absolute judgment and relative judgment?
Absolute judgment is an evaluation method in which an output is assessed independently, often by assigning it a score, rather than comparing it with another output. A method of assessment in which judges compare outputs with one another rather than assigning each an independent score.
What is absolute judgment a kind of?
Absolute judgment is a kind of judgment format. A form in which a judgment is expressed, such as a binary choice, a scalar value, or a comparison between outputs.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.

