liveBook · Manning

Extrinsic hallucination

As explained in LLM Evaluation and Alignment, The Foundational Ideas

An extrinsic hallucination is a generated utterance that adds information not directly inferable from or contained in the source material.

An extrinsic hallucination adds information that the source does not contain or directly imply. In a closed-domain task, it violates source fidelity: the added claim may be plainly false, or may happen to be correct but remain unverifiable from the supplied material. section 5.1.2 section 5.5

Classify the output by contextual grounding and factual alignment:

  • Extrinsic but correct: adds a true claim absent from the source, making the response risky because source-based verification cannot establish it. section 5.1.2
  • Extrinsic and misaligned: adds a plausible but unsupported or fabricated claim, such as attributing revenue growth to an AI product launch absent from the source. section 5.1.2
  • Detect it through claim decomposition and context verification, NLI, LLM-as-a-Judge, or consistency checks such as SelfCheckGPT; consistency alone misses stable errors. section 5.5

Claim-level faithfulness check

Suppose an answer makes six factual claims about an earnings report, but only three appear in the retrieved context. Faithfulness is 3/6 = 0.50, and the three unsupported claims are extrinsic hallucinations. Track this metric to distinguish generation failures from retrieval gaps. section 6.3.1 section 6.5

Figure 5.3 The LLM Reliability Quadrant. Contextual grounding (vertical axis) asks whether the output stays within the provided sources. Factual alignment (horizontal axis) asks if the output is correct according to verifiable evidence. The safest output is both grounded and aligned. The riskiest output is neither grounded nor aligned. The mixed-signal quadrants are operationally important because the output can look based on source but wrong, or look correct but not grounded.

Use the reliability quadrant to compare contextual grounding with factual alignment and identify unsupported generation as a distinct failure mode. section 5.1.2

How LLM Evaluation and Alignment, The Foundational Ideas covers it

How deep it goes
Explains grounding and alignment categories, claim-level verification, detection methods, and faithfulness scoring.
Examples
Uses unsupported headquarters and AI-launch claims, plus six-claim earnings-report verification.
What it assumes
Assumes a provided source of truth and distinguishes closed-domain boundaries from open-domain reliance on internal or external knowledge.
What you can do after
Diagnose unsupported claims by decomposing answers, checking retrieved context, and interpreting faithfulness alongside retrieval metrics.
How it connects
Connects contextual grounding, factual alignment, intrinsic hallucination, claim decomposition, faithfulness, and RAGAS.

Also explained in

Compare how 2 books explain extrinsic hallucination →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.