Factuality
Factuality is the degree to which claims in content accurately represent externally verifiable facts.
Introduction to Generative AI, Second Edition
Factuality is the degree to which generated content accurately represents facts about the world.
- How it is used
- Highlights the risk of relying on plausible LLM output in professional work.
- How deep it goes
- Explains the limitation through a legal case but does not provide a factuality measurement method.
- Examples
- Uses a legal brief containing invented lawsuits to illustrate false generated claims.
- What you can do after
- Evaluate generated claims by checking them against reliable sources.
Read how Introduction to Generative AI, Second Edition explains factuality →
Machine Learning Platform Engineering
Factual accuracy is the degree to which generated content agrees with verifiable facts.
- How it is used
- Guides LLM evaluation when probabilistic outputs make exact-match testing inadequate.
- How deep it goes
- Defines factual accuracy and its evaluation role, but stops short of prescribing a concrete scoring algorithm.
- Examples
- Contrasts exact assertions with semantic evaluation and relates results to temperature, top-K, and top-P sampling.
- What you can do after
- Evaluate LLM responses with semantic and factual criteria instead of exact-string assertions.
Read how Machine Learning Platform Engineering explains factuality →
Also covered in
- Spring AI in Action
- Building LLM Applications with DSPy
- Machines that ThinkIn early access (MEAP)
- Build an Advanced RAG Application (From Scratch)In early access (MEAP)
- AI Model EvaluationIn early access (MEAP)
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- LLM-as-a-judgeLLM-as-a-judge is an evaluation technique in which a language model assesses or compares outputs according to specified criteria.
- retrieval-augmented generationRetrieval-augmented generation is a technique that retrieves relevant information from external sources and supplies it as context to a generative model when producing an output.
- fine-tuningFine-tuning is the continued training of a pretrained model on targeted data or objectives to adapt its parameters or behavior to a task, domain, or use.
Related concepts
- LLM-as-a-judgemeasured by · LLM-as-a-judge is an evaluation technique in which a language model assesses or compares outputs according to specified criteria.
- evaluationmeasured by · Evaluation is the systematic assessment of a system, model, process, or outcome against relevant criteria or objectives using evidence.
- ground truthmeasures · Ground truth is reference information treated as correct for assessing the accuracy or correctness of predictions, labels, or other outputs.
- checklistused by · A structured list of tasks, questions, or criteria used to guide an activity or systematically verify that relevant requirements have been addressed.
Questions about factuality
What is factuality?
Factuality is the degree to which claims in content accurately represent externally verifiable facts.
Which Manning books explain factuality?
Introduction to Generative AI, Second Edition and Machine Learning Platform Engineering.
What should I know before learning factuality?
Manning books that teach factuality usually introduce language model, LLM-as-a-judge, retrieval-augmented generation and fine-tuning first.
What is factuality also called?
Factuality is also called factual accuracy or factual correctness.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.

