liveBook · Manning

Hallucination

As explained in AI Governance

An AI hallucination is an output that presents fabricated, unsupported, or false information as though it were valid.

A generative AI system can produce an answer that sounds confident but contains fabricated or false information. This is a hallucination: the output presents an unsupported claim as fact. Plausible wording does not establish accuracy, and a false claim can be mixed with real details. section 1.1 section 5.4

A fabricated claims history

An insurance chatbot tells a customer that she has had two rear-end collisions and warns that another could trigger a policy review. In fact, her recent accident was her first in a decade. The chatbot has presented a fictional claims history as fact. section 5.4

False claims about identifiable people can be personal data even when inaccurate. They may require correction, and harmful processing may be unlawful regardless of intent. If such claims influence significant decisions, such as job application rejection, the consequences extend beyond an incorrect answer. section 5.4

Hallucinations can also invent legal facts or citations, or combine real personal details with defamatory claims. Overreliance magnifies the harm: a convincing answer can mislead the person who uses it. Generative AI can produce such errors quickly and at scale. section 1.1 section 5.4

Table 1.1 Why Generative AI Breaks Traditional Control Models (Continued)
Pre-deployment assurance
Tests, code review, static analysis, red-teaming (ethical hackers)
Role-specific training, competency certification, scenario-based drills
Safety and bias testing, adversarial prompt suites, red-teaming, interpretability tools (e.g. probing)
Post-launch controls
Logging, health checks, intrusion detection, WAF, auto-rollback on error rate spike
Supervisors, peer review, escalation protocols
Output filters, rate limiting, live metrics on toxicity and drift, anomaly detection on embeddings
Failure detection & attribution
Clear (stack traces, logs, input replay)
Mixed (relies on human reporting  or observation)
Opaque (failures can appear valid; root cause buried in model weights)
Memory / learning capability
None (unless explicitly coded)
Present (experience, judgment, learning over time)
Present (training data, fine-tuning, context memory)

Compare pre-deployment and post-launch controls with the difficulty of detecting AI failures that appear valid. These general controls do not by themselves establish that a particular claim is true. section 1.1

How AI Governance covers it

How deep it goes
Explains plausible fabrication and its privacy consequences, while deferring hallucination and overreliance remediations.
Examples
Uses a fabricated insurance claims history, a false criminal accusation, and AI-invented legal facts or citations.
What you can do after
Evaluate the risks of relying on plausible AI claims, especially when they concern identifiable people or inform significant decisions.
How it connects
Connects hallucination to personal data, automated decision-making, overreliance, and data protection.

Also explained in

Compare how 37 books explain hallucination →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Governance. The section links point to the parts of the book it draws on.