liveBook · Manning

Intrinsic hallucination

As explained in LLM Evaluation and Alignment, The Foundational Ideas

An intrinsic hallucination is an output that is grounded in the provided source but factually misaligned with it.

An intrinsic hallucination is a generated response that stays grounded in the provided source but misrepresents facts shown there. It combines contextual grounding with factual misalignment. section 5.1.2 section 5.5

Classify source-grounded outputs by two independent questions:

  • Does the response remain within the provided source, or introduce unsupported information? section 5.1.2
  • Does each claim match verifiable facts, or contradict them? section 5.1.2
  • Intrinsic hallucination occupies the grounded-but-misaligned quadrant; extrinsic hallucination is both ungrounded and misaligned. section 5.5

Financial report example

A source states, “Q3 revenue was $2.3 billion, a 15% increase year-over-year.” Writing “Q3 revenue was $2.3 billion, a 5% increase year-over-year” uses the source’s data but changes a fact. Comparing output claims with source claims exposes the contradiction. section 5.1.2

Figure 5.3 The LLM Reliability Quadrant. Contextual grounding (vertical axis) asks whether the output stays within the provided sources. Factual alignment (horizontal axis) asks if the output is correct according to verifiable evidence. The safest output is both grounded and aligned. The riskiest output is neither grounded nor aligned. The mixed-signal quadrants are operationally important because the output can look based on source but wrong, or look correct but not grounded.

Use the reliability quadrant to separate contextual grounding from factual alignment and identify mixed-signal failures. section 5.1.2

In retrieval-augmented generation, low faithfulness with strong context scores indicates a generation problem, including intrinsic hallucination from misreading retrieved evidence. section 6.5

How LLM Evaluation and Alignment, The Foundational Ideas covers it

How deep it goes
Defines the grounded-but-misaligned category, contrasts neighboring quadrants, and demonstrates claim-level contradiction checking.
Examples
Uses a financial revenue statement whose 15% growth rate is incorrectly changed to 5%.
What it assumes
Assumes only a provided source and basic distinctions between grounding and factual correctness.
What you can do after
Diagnose grounded factual errors by comparing generated claims against source claims and interpreting low-faithfulness RAGAS results.
How it connects
Connects contextual grounding, factual alignment, extrinsic hallucination, faithfulness, claim decomposition, and RAGAS.

Also explained in

Compare how 2 books explain intrinsic hallucination →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.