liveBook · Manning

Hallucination

As explained in Retrieval Augmented Generation, The Foundational Ideas

A generated claim that contradicts the evidence or source context available to the generation system.

A customer-facing assistant can give a confident answer that its evidence does not support. The result may look useful while damaging trust: Air Canada’s assistant failure illustrates the reputational cost. A hallucination is a plausible generated claim that is factually wrong or ungrounded in the available source material. Unlike a software error that announces itself, it can fail silently. section 1.3 section 1.3.2

The distinction between error types helps locate the failure:

  • Factual hallucinations contradict reality; citation hallucinations invent sources or links that do not resolve. section 1.3.2
  • Intrinsic hallucinations contradict retrieved text; extrinsic hallucinations add details absent from it. A claim can therefore sound reasonable without being supported by the supplied evidence. section 1.3.2
  • Logical hallucinations contradict the answer’s own reasoning; temporal hallucinations confuse historical and current information. section 1.3.2

When most of an answer is right

A legal research answer cited a real case but attributed it to a fictional judge. The accurate surrounding details made the invented detail harder to notice. In an open-domain question-answering comparison, a model answering from its parameters alone gave “Vancouver” as the largest city in the Pacific Northwest; a retrieval-augmented system combined passages about the region and Seattle to answer “Seattle.” section 1.3.2 section 4.5.1

Retrieval-augmented generation addresses this risk by locating external information and using it to generate an answer. Retrieval is not itself verification: diagnose whether relevant evidence was found, whether the answer used it faithfully, and whether each consequential claim is supported. Larger models and longer prompts alone do not remove the problem; polished fabrication can be difficult to detect, while excessive context can obscure key facts. section 1.3.2 section 1.3.4 section 1.6

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Defines the failure and classifies six forms, but does not provide a step-by-step verification procedure.
Examples
Uses a fabricated legal attribution and a Pacific Northwest question answered differently by parameter-only and retrieval-augmented systems.
What it assumes
Defines hallucination from zero while using language models, retrieval, and generation as the surrounding concepts.
What you can do after
Diagnose/debug with it by separating unsupported claims from retrieval and answer-synthesis failures.
How it connects
Connects hallucination to retrieval-augmented generation, the knowledge boundary, source grounding, and factual consistency.

Also explained in

Compare how 37 books explain hallucination →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.