A legal brief can sound convincing while citing lawsuits that never happened. Factuality is the degree to which generated content accurately represents facts about the world. Plausible writing is not, by itself, evidence that a claim is true. section 5.8
Large language models (LLMs) can summarize information and produce human-like text, but they generally cannot ensure factuality or accuracy. Their fluent responses can lead users to accept claims without checking them against reliable sources. Verification is especially important in fields requiring specialized knowledge, such as law, medicine, and finance. section 5.8
Mata v. Avianca
An attorney used ChatGPT to help compile a legal brief. It invented supporting arguments based on lawsuits that had never occurred. The attorney said he had not intended to deceive the court and had been unaware that the tool’s content could be false. section 5.8

