liveBook · Manning

Hallucination

As explained in Build an Advanced RAG Application (From Scratch)

A hallucination is a model-generated statement or content that is factually incorrect or nonsensical while appearing coherent.

A hallucination is a model-generated statement or content that is factually incorrect or nonsensical while appearing coherent. LLMs can produce hallucinations when confidently generating information not explicitly learned or responding to out-of-distribution prompts, making validation important in critical applications. section 1.3

LLMs predict likely next tokens from training data rather than retrieving documents from a current, verified knowledge base. They can therefore produce plausible but outdated, inaccurate, or fictional details. Retrieval-augmented generation reduces this risk by retrieving relevant information with encoder models before decoder models generate a grounded response. section 5.3.1

A fabricated answer about Mars

Prompt: “Who was the first person to walk on Mars?” Response: “John Smith was the first person to walk on Mars in 2025.” The answer is coherent but fabricated because no human has walked on Mars. section 5.5

Figure 5.16 A great example of a hallucinated response generated by ChatGPT 4.0. It generated a factually incorrect response to the questions about Traversaal.ai – given it was founded in June 2023, by me 😊

Use the figure as a concrete visual example of a decoder producing a factually incorrect response about an organization. section 5.5

How Build an Advanced RAG Application (From Scratch) covers it

How deep it goes
Explains the definition, token-prediction mechanism, examples, and RAG-based mitigation without detailing validation algorithms.
Examples
Uses a fictional first-person-on-Mars answer and inaccurate hotel-information generation as concrete examples.
What you can do after
Apply retrieval-augmented generation to ground generated responses in retrieved, relevant information.
How it connects
Connects hallucination with LLMs, prompting, search engines, encoder models, decoder models, and retrieval-augmented generation.

Also explained in

Compare how 37 books explain hallucination →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build an Advanced RAG Application (From Scratch). The section links point to the parts of the book it draws on.