A hallucination is an AI output that is wrong, confidently presented as correct, and liable to appear unpredictably. It may invent facts, misread real information, or give an incorrect solution. section 2.1 section 2.16
A plausible answer that is simply false
Asked for real books containing very long words, GPT-4o supplied convincing titles and author names that did not exist, while assuring the user they were real. The same pattern appears in legal research when invented court cases are submitted as precedent. section 2.2
Hallucinations commonly take three forms:
- Made-up facts: the model fabricates books, citations, names, or other information. section 2.2
- Misinterpretation: the information exists, but the model extracts it without understanding its context, including from retrieved documents. section 2.3
- Incorrect solutions: the model performs a task wrongly while claiming success, such as drawing a beard it was told to omit or mishandling a river puzzle. section 2.4
Two traits make hallucinations especially dangerous. Overconfidence hides uncertainty and can persist even after the user points out an error. Unpredictability means a tiny change in wording can turn a correct answer into a wrong one, or produce contradictory answers to the same question. section 2.5 section 2.6
The problem is deeper than a minor software bug. Language models optimize next-token prediction, not truth itself; their internal world models can also rely on shortcuts or memorized patterns rather than robust understanding. Retrieval-augmented generation may provide factual documents without guaranteeing correct interpretation. section 2.10 section 2.8
Treat hallucination risk as a product decision, not merely a model-quality metric. Incorrect hotel-review translations may be tolerable, while wrong refunds, drive-through orders, legal advice, or vehicle decisions can damage users and destroy trust. Assess the consequences early, then redesign the task, add mitigation or detection, or avoid AI where errors are unacceptable. section 2.14 section 4.3 section 2.15
Figure 2.1 An example of AI drawing what it is explicitly told not to
Use the figure to inspect an image-generation failure in which the model includes the feature the prompt explicitly prohibits. section 2.4
