4 Advancing trust & minimizing hallucinations with retrieval augmented generation

 

This chapter covers

  • Retrieval augmented generation (RAG) and how it overcomes limitations of stand-alone LLMs
  • Core components of a RAG architecture - retrievers, generators, orchestrators
  • Indexing and structuring knowledge sources to enable relevant passage retrieval
  • Building sample RAG systems with LangChain for simplifying orchestration

In our prior chapter, a world of possibilities opened in constructing conversational AI through prompting - carefully crafting input texts to large language models (LLMs) to shape helpful, eloquent chatbot responses. However, despite the disruptive potential, major gaps remain in flexibility for real-world assistance.

Figure 4.1 Basic prompting through interacting with LLM directly. [1]

With basic prompting (shown in figure 4.1), LLMs have no direct means to access live external data streams beyond their training corpora. However, allowing chatbots to incorporate dynamic knowledge is crucial, as we see below for our retail ecommerce chatbot. Could prompting alone enable a shopper asking:

Do you have the OLED 55 inch TV with integrated soundbar in stock for same day delivery?

Figure 4.2 Asking an LLM (Claude 2) about a specific question related to inventory systems

4.1 What is Retrieval Augmented Generation?

4.2 RAG System Architecture

4.3 Reducing Hallucinations with RAG

4.4 Data Preparation and Reliable Indexing for RAG Systems

4.4.1 Structuring Product Catalogs

4.4.2 Creating the searchable vector index

4.4.3 Implementing a FAISS Index

4.5 Building an Effective RAG system

4.5.1 Processing User Queries

4.5.2 Crafting an Accurate Response Using an LLM

4.6 Evaluating and Optimizing RAG Systems

4.6.1 The Role of Evaluator LLMs in Assessing Hallucinations

4.6.2 Crafting Evaluation Datasets

4.6.3 Practical Metrics for RAG Evaluation

4.6.4 Reducing Hallucinations by Filtering Retrievals and Leveraging Metadata

4.6.5 Embedding chunk sizes and model choice

4.7 Advanced Techniques for Improving RAG

4.8 Introduction to LangChain

4.8.1 Building a RAG System for an Ecommerce Chatbot with LangChain

4.8.2 Challenges and Adaptations of the Chatbot

4.9 Summary

4.10 References