Fabricated content is information or media created or altered to appear authentic even though it does not represent genuine facts or events. Generative AI can produce fluent, convincing material that is factually inaccurate or entirely fictional, making confident output unsafe to trust without verification. section 5.10 section 1.3
Invented legal cases
A lawyer submitted a brief containing six fictitious court cases, complete with fabricated quotations and citations, after ChatGPT affirmed that one nonexistent case was genuine and available in legal databases. The incident shows how plausible fabrication can become consequential when users trust an AI system without checking its claims. section 5.10
Repeatedly changing personal facts
When asked who Akli Adjaoute was, ChatGPT produced several different incorrect biographies, including an invented company, awards, education, and employment history. It returned an accurate response only after five attempts, demonstrating that fluent answers can vary while remaining false. section 5.3.2
Figure 5.1 ChatGPT prompt, which resulted in incorrectly identifying the author as the founder and CEO of a company he was not affiliated with.
The figure shows a prompt that caused ChatGPT to assign a person to an unrelated company, making the mechanism of fabricated identity claims concrete. section 5.3.2
Fabricated content creates practical risks when it is:
- Presented as legal, medical, or other authoritative guidance without independent review. section 5.10
- Spread through social media, deepfakes, or realistic recordings to deceive or manipulate people. section 1.3
- Accepted blindly because the output sounds coherent, especially by nontechnical users. section 1.3
