9 Creating an agentic RAG-based chatbot
This chapter covers
- Using DSPy for retrieval-augmented generation
- Agents and the ReAct methodology
- Agentic retrieval-augmented generation
- DSPy’s support for memory and long-running conversations
So far, we’ve covered the most important ideas in DSPy and described how to create, evaluate, and optimize DSPy-based applications. We’re now ready to create more sophisticated and realistic systems that you’ll likely need in research or work environments. These include retrieval-augmented generation (RAG)–based applications, agents, and systems that can maintain conversations over long periods of time. DSPy supports all of these systems and can combine them to create powerful applications, including fully functional chatbots.
We look first at RAG applications, which allow us to work with LMs so that they can respond to queries even when doing so requires information they didn’t see during training. For example, a company chatbot can accept questions from customers about the company related to its policies, products, or services, such as “Can I return a coffee maker I bought if it’s unused but the box is opened?” The chatbot may make one or more language model (LM) calls to answer these questions, but the LMs won’t, on their own, have any way to know the answer.