A plausible answer can mix supported facts with invented details. Faithfulness measures the proportion of an answer’s claims supported by evidence. A score near 1 means most claims are supported; a score near 0 means few are. It is a measure of grounding, not a complete measure of answer quality. section 1.1.3 section 6.3.1
To calculate faithfulness against a source:
- Decompose the answer into individual, independently verifiable claims. section 5.1.4 section 6.3.1
- Check each claim against the source and mark whether the evidence supports it. section 5.1.4 section 6.3.1
- Divide the number of supported claims by the total number of claims. section 6.3.1
A financial research answer
An answer makes six claims about a company. The retrieved earnings report supports its revenue, growth rate, and cloud-services explanation, but does not mention European expansion, a new CEO, or international revenue growth. Faithfulness is 3/6, or 0.50. The unsupported details may sound plausible, but the retrieved evidence does not establish them. section 6.3.1
Choose the verification target carefully. In a closed-domain task, such as document question answering, claims can be checked against the supplied source. In an open-domain task, there may be no single grounding document; external evidence or reference-free consistency checks may be needed. Consistency alone can miss a false claim that the model repeats reliably. section 5.1.1 section 5.5
Figure 5.3 The LLM Reliability Quadrant. Contextual grounding (vertical axis) asks whether the output stays within the provided sources. Factual alignment (horizontal axis) asks if the output is correct according to verifiable evidence. The safest output is both grounded and aligned. The riskiest output is neither grounded nor aligned. The mixed-signal quadrants are operationally important because the output can look based on source but wrong, or look correct but not grounded.
Use the reliability quadrant to distinguish source grounding from factual correctness: an unsupported claim can still happen to be true, and a grounded claim can still be wrong. section 5.1.2
In retrieval-augmented generation, faithfulness compares the answer with retrieved context. Pair it with answer relevance and retrieval measures: a faithful answer can still be incomplete if retrieval missed important information. Conversely, relevant retrieved documents do not guarantee that the answer uses them faithfully. section 6.1.2 section 6.3
Interpret the score alongside the claims and evidence. Ambiguous statements, context-dependent assertions, inference chains, and half-truths make both claim extraction and support judgments difficult. Define trusted sources and evaluation criteria explicitly, and use human safeguards for high-stakes decisions. section 1.1.3 section 5.1.2 section 5.4.5










