chapter three

3 Grounding outputs with RAG

 

This chapter covers

  • Seeing how retrieval-augmented generation (RAG) overcomes the limitations of stand-alone models
  • Examining the core components of an RAG architecture: retrievers, generators, and orchestrators
  • Indexing and structuring knowledge sources to enable relevant passage retrieval
  • Building sample RAG systems with LangChain to simplify orchestration

Chapter 2 opened a world of possibilities for constructing conversational AI through prompting: carefully crafting input texts to large language models (LLMs) to shape helpful, eloquent chatbot responses. But despite the disruptive potential, major gaps remain in flexibility for real-world assistance.

With basic prompting alone (figure 3.1), LLMs have no direct way to access live external data streams beyond their training corpora. Some AI products built on top of LLMs, such as ChatGPT and Claude, can call external tools like web search to fetch fresh information. But that capability comes from the application orchestrating the LLM, not from the model itself. Allowing chatbots to incorporate dynamic knowledge is crucial, as we’ll see with our retail e-commerce chatbot in this chapter. Could prompting alone support a shopper asking this question?

Figure 3.1 Basic prompting through interacting with an LLM directly [1]
figure

Do you have the OLED 55 inch TV with integrated soundbar in stock for same day delivery?

3.1 What is RAG?

3.2 RAG system architecture

3.2.1 The role of the retriever

3.2.2 The factual generators

3.2.3 RAG system flow

3.2.4 Example: E-commerce shopping assistant

3.3 Reducing hallucinations with RAG

3.3.1 Grounding the language model

3.3.2 Grounding an e-commerce product query

3.3.3 Advantages of RAG in reducing hallucinations

3.3.4 Critical applications

3.3.5 Enhancing transparency

3.4 Data preparation and reliable indexing for RAG systems

3.4.1 Introducing LangChain

3.4.2 Structuring product catalogs