2 Understanding and measuring hallucinations in LLMs

 

This chapter covers

  • Types of hallucinations in LLMs
  • Identifying and measuring hallucinations
  • Mitigating hallucinations in LLMs

Imagine you're a lawyer working on an important case. You're short on time, so you decide to use an AI legal assistant to help with research. You give the AI some prompts, and it quickly generates a draft full of convincing arguments and legal precedents. Impressed, you include parts of the AI's output in your own brief and submit it to the court.

But there's a big problem: some of the legal cases the AI referred to don't actually exist. They're completely made up. By using this "hallucinatory" content without double-checking it, you've accidentally misled the court and put your own reputation at risk.

This isn't just a hypothetical scenario. In 2023, a law firm in New York had to pay a $5,000 fine for doing exactly this—submitting a brief with fake cases generated by ChatGPT. The lawyer who used ChatGPT didn't catch that some of the cases it cited weren't real. This shows how serious the consequences of AI hallucinations can be.

As language AI models like GPT have become more advanced and more widely used, hallucinations have emerged as a major challenge. Hallucinations occur when AI systems generate content that is untrue, inconsistent, or just plain made-up. This can lead to the spread of fake information, incorrect medical diagnoses, bad financial advice, and many other potential harms.

2.1 What are Hallucinations Exactly?

2.1.1 Unpacking the Use of "Hallucination" in AI

2.1.2 Types of Hallucinations

2.1.3 Why Do Hallucinations Occur?

2.2 How to Identify and Measure Hallucinations

2.2.1 Four Steps to Identify and Measure Hallucinations

2.2.2 ROUGE Metric: News Summarization Example

2.2.3 QA-Based Metrics

2.2.4 LLM as a Judge: A Holistic Approach to Hallucination Detection

2.2.5 Human Evaluation

2.2.6 Red Teaming: Stress-Testing LLMs for Robustness

2.2.7 Using Arize and the Phoenix Framework for Hallucination Detection

2.3 Mitigating Hallucinations

2.3.1 Selecting the Right Model

2.3.2 Data-Related Methods to Improve Relevance and Reduce Hallucinations

2.3.3 Post-Processing to Lower Hallucinations

2.3.4 Retrieval-Augmented Generation to Improve Grounding

2.3.5 Prompting Techniques to Lower Hallucinations

2.3.6 Mitigating Hallucinations Through Produce Design and User Interaction

2.4 Summary