liveBook · Manning

Detection

As explained in LLM Evaluation and Alignment, The Foundational Ideas

Hallucination detection is the identification of generated claims or text that contain false, unsupported, or fabricated information.

Hallucination detection identifies generated claims that are false, unsupported, or fabricated. It targets the gap between fluent model output and factual accuracy, which lexical overlap, semantic similarity, and general helpfulness scores may miss. section 5.0 section 5.1.3

Structure detection around two capabilities and choose methods according to available evidence:

  • Extract claims by splitting a response into sentences or decomposing it into atomic, independently verifiable claims. section 5.1.4 section 5.4.1
  • Verify claims with self-consistency sampling for open-domain answers, NLI against retrieved evidence, or claim-level verification for auditable systems. section 5.1.4 section 5.4.1
  • Use SelfCheckGPT when no trusted reference or external knowledge base exists; compare multiple independent generations to find inconsistent content. section 5.0 section 5.2.5
  • Aggregate sentence- or claim-level results into faithfulness, maximum hallucination, weighted faithfulness, or hallucination-percentage metrics. section 5.4.1
  • Log detections, route them to human review, block responses, or trigger regeneration according to operational risk and latency. section 5.4.1

Mixed factual and fabricated support response

A response may correctly state that order #4521 shipped on March 3rd via FedEx while inventing a premium-membership policy. Detection must verify the claims separately rather than trusting fluent context. In production, a faithfulness failure can act as a gate that blocks further helpfulness and tone scoring. section 5.0 section 5.4.4

Figure 5.4 An example of an agent assistant outputs answers in a very fluent and convincing manner, but the answer is incorrect.

Use the figure to recognize the central diagnostic failure: confident, fluent output can still be incorrect. section 5.1.3

Detection is a monitoring layer, not a prevention method. It can expose systematic errors for alignment work, but it may miss consistent false beliefs, requires extra generations for self-consistency, and often flags whole sentences instead of the faulty sub-claim. section 5.2.5 section 5.4.5

How LLM Evaluation and Alignment, The Foundational Ideas covers it

How deep it goes
Decomposes detection into extraction, verification, aggregation, and action, while deferring detailed mitigation and alignment methods.
Examples
Uses customer-support policies, mixed factual claims, fabricated authorship, and fluent incorrect assistant instructions as diagnostic examples.
What it assumes
Assumes generated responses and evidence sources, while defining faithfulness, claim extraction, and claim verification as foundational ideas.
What you can do after
Diagnose generated responses with claim-level checks, aggregate faithfulness metrics, and production actions such as review, blocking, or regeneration.
How it connects
Connects hallucination detection to faithfulness, claim verification, SelfCheckGPT, NLI, retrieval-augmented generation, and alignment.

Also explained in

Compare how 2 books explain detection →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.