Claim verification checks whether each factual claim is supported by available evidence. It turns a generated response into independently testable assertions and exposes unsupported details instead of hiding them behind fluent prose. section 5.1.4 section 5.3.1
Use a claim-verification workflow:
- Extract atomic claims, making each statement independently verifiable. section 5.3.1
- Select evidence appropriate to the claim: retrieved documents, structured knowledge bases, or web search. section 5.3.1
- Classify each claim as supported, contradicted, or lacking evidence. section 5.3.1
- Aggregate the results into faithfulness, hallucination counts, or importance-weighted quality gates. section 5.4.1 section 6.3.1
- Route failures to logging, human review, response blocking, or regeneration. section 5.4.1
Financial RAG response
Decomposing an answer into six claims about revenue, growth, cloud services, European expansion, a new CEO, and international revenue yields three supported and three unsupported claims. Faithfulness is therefore 3/6 = 0.50; the unsupported claims reveal fabricated details introduced when retrieved documents leave information gaps. section 6.3.1
Listing 5.4 Claim extraction using an LLM
CLAIM_EXTRACTION_PROMPT = """Extract all factual claims from the following text. Each claim should be a single, independently verifiable statement. Return one claim per line.
Text: {text}
Claims:"""
def extract_claims(text: str, model: str = “gpt-5.4”) -> list[str]:
response = client.chat.completions.create(
model=model,
messages=[{
"role": "user",
"content": CLAIM_EXTRACTION_PROMPT.format(text=text)
}],
temperature=0
)
claims = response.choices[0].message.content.strip().split(“\n”)
return [c.strip(“- “).strip() for c in claims if c.strip()]
Shows an LLM prompt and function for extracting independently verifiable claims. section 5.3.1
Listing 5.5 Claim verification against reference documents
VERIFICATION_PROMPT = """Given the following reference document and a factual claim, determine if the document SUPPORTS the claim, CONTRADICTS the claim, or provide NO EVIDENCE for the claim.
Reference Document:
{document}
Claims: {claim}
Verdict (SUPPORTS / CONTRADICTS / NO EVIDENCE):"""
def verify_claim(claim: str, documents: list[str], model: str = “gpt-5.4”) -> dict:
verdicts = []
for doc in documents:
response = client.chat.completions.create(
model=model,
messages=[{
"role": "user",
"content": VERIFICATION_PROMPT.format(document=doc, claim=claim)
}],
temperature=0,
max_tokens=10
)
verdict = response.choices[0].message.content.strip()
verdicts.append(verdict)
supported = sum(1 for v in verdicts if “SUPPORTS” in v)
contradicted = sum(1 for v in verdicts if “CONTRADICTS” in v)
return {
“claim”: claim,
“supported”: supported,
“contradicted”: contradicted,
“no_evidence”: len(verdicts) - supported - contradicted,
“is_hallucination”: contradicted > 0 or supported == 0 #A
}
Shows reference-document verification with SUPPORTS, CONTRADICTS, and NO EVIDENCE verdicts. section 5.3.1
Listing 5.6 Computing faithfulness from verified claims
def faithfulness(text: str, reference_docs: list[str]) -> dict:
claims = extract_claims(text)
results = []
for claim in claims:
verification = verify_claim(claim, reference_docs)
results.append(verification)
supported = sum(1 for r in results if not r[‘is_hallucination’])
total = len(results)
faithfulness = supported / total if total > 0 else 1.0
return {
“faithfulness”: faithfulness,
“total_claims”: total,
“supported_claims”: supported,
“hallucinated_claims”: total - supported,
“details”: results #A
}
Shows how verified claims become faithfulness, support, hallucination, and per-claim detail metrics. section 5.3.1








