chapter five

5 Detecting and quantifying hallucinations

 

This chapter covers

  • Hallucination types and why they resist simple detection
  • SelfCheckGPT’s self-consistency-based approach for zero-resource hallucination detection
  • Claim decomposition and verification as a structured detection framework
  • Building hallucination detection pipelines for production systems

In chapter 1, we introduced hallucination as AI’s most dangerous failure mode. The generated content is fluent and confident, but completely fabricated and unsupported. We showed that faithfulness, the ratio of supported claims to total claims, provides a quantitative framework for measuring it. But we left a critical question unanswered: how do you actually detect which claims are hallucinated?

Consider this production scenario. Your retrieval augmented generation (RAG) powered customer support system generates a response: “Your order #4521 shipped on March 3rd via FedEx and is expected to arrive by March 7th. Our premium members receive free expedited shipping on all orders over $50.” The first sentence can be verified against your order database. The second sentence sounds like a company policy, but your company has no premium membership tier. A human reading this response might not catch the fabrication as it fits within the expected context. How do you build a system that catches it automatically, at scale, without a human reviewing every response?

5.1 An LLM reliability quadrant

5.1.1 Detection context: Closed-domain vs open-domain tasks

5.1.2 Dimensions of reliability

5.1.3 Why hallucinations resist simple detection

5.1.4 Connecting to the faithfulness metric

5.2 SelfCheckGPT: Consistency as a hallucination signal

5.2.1 Self-consistency

5.2.2 The SelfCheckGPT framework

5.2.3 Four variants of measuring consistency

5.2.4 Sampling in AI Systems

5.2.5 SelfCheckGPT results and limitations

5.3 Complementary hallucination detection methods

5.3.1 Claim decomposition and verification

5.3.2 Uncertainty-based detection

5.4 Building hallucination detection pipelines

5.4.1 Pipeline architecture

5.4.2 Online vs offline hallucination detection

5.4.3 Calibrating detection thresholds

5.4.4 Integration with evaluation systems

5.4.5 When detection is not enough

5.5 Summary