Factuality
Factuality is the degree to which claims in content accurately represent externally verifiable facts.
Introduction to Generative AI, Second Edition
Factuality is the degree to which generated content accurately represents facts about the world.
- How it is used
- Frames factuality as a reliability concern when large language models produce professional writing.
- How deep it goes
- Distinguishes plausible prose from accurate claims, but gives no factuality score or formal testing procedure.
- Examples
- Uses a ChatGPT-assisted legal brief citing nonexistent lawsuits in Mata v. Avianca.
- What you can do after
- Evaluate/understand it by checking plausible generated claims against reliable sources before relying on them professionally.
Read how Introduction to Generative AI, Second Edition explains factuality →
Machine Learning Platform Engineering
Factual accuracy is the degree to which generated content agrees with verifiable facts.
- How it is used
- Guides LLM evaluation when probabilistic outputs make exact-match testing inadequate.
- How deep it goes
- Defines factual accuracy and its evaluation role, but stops short of prescribing a concrete scoring algorithm.
- Examples
- Contrasts exact assertions with semantic evaluation and relates results to temperature, top-K, and top-P sampling.
- What you can do after
- Evaluate LLM responses with semantic and factual criteria instead of exact-string assertions.
Read how Machine Learning Platform Engineering explains factuality →
Related concepts
- evaluatormeasured by · An evaluator is a model or component that assesses outputs against specified criteria and produces judgments, scores, comparisons, or feedback.
- evaluationmeasured by · Evaluation is the systematic assessment of a system, result, or approach against relevant criteria, evidence, or objectives.
- checklistused by · A structured list of tasks, questions, or criteria used to guide an activity or systematically verify that relevant requirements have been addressed.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.