liveBook · Manning

Precision

As explained in Introduction to Generative AI, Second Edition

Precision is the proportion of positive predictions that are correct among all positive predictions made by a classifier.

Precision measures how often a classifier’s positive predictions are correct: among all positive predictions, it counts the share that match the intended positive label. In language-model evaluation, it helps assess whether an automated judge agrees with expert annotations. High precision means positive judgments are usually trustworthy, though precision alone does not show how many true positives were missed. section 7.10

For example, an LLM judge can label real-estate descriptions for enthusiasm or positivity and then be compared with human labels. Precision and recall are preferable to relying only on overall accuracy when positive and negative examples are unevenly represented. section 7.10

How Introduction to Generative AI, Second Edition covers it

How deep it goes
Defines positive-prediction correctness and applies it to classifier evaluation, without detailing calculation steps or threshold selection.
Examples
Uses an LLM judge scoring enthusiasm and positivity in generated real-estate listing descriptions.
What it assumes
Assumes binary labels and classifier predictions while treating expert annotation as the comparison reference.
What you can do after
Evaluate annotation prompts and compare future model changes against expert-labeled judgments.
How it connects
Connects precision with recall, accuracy, binary classification, human annotation, and LLM judges.

Also explained in

Compare how 20 books explain precision →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Introduction to Generative AI, Second Edition. The section links point to the parts of the book it draws on.