What will I learn in chapter 9 of Building LLM Applications with DSPy?

The introduction explains how retrieval, tool-using agents, and conversation history help DSPy chatbots answer questions and carry out requests. After reading it you should be able to: Explain how RAG lets a chatbot answer questions requiring information its language model did not see during training. Describe how it finds relevant documents and supplies them with the question as context for the model. Distinguish the language model's role in guiding tool use from the agent's role in executing tools to respond to a user's request. Explain that WixQA provides Wix customer questions, documents that support answers, and expert-written answers assumed to be correct, which a RAG application aims to replicate. Describe the two WixQA subsets used here and how article IDs in a question-and-answer row identify supporting knowledge base entries, sometimes more than one. Explain how a DSPy RAG module first gathers relevant data and then passes that data with the original query to an LM for an answer. Describe how article titles and a customer question are represented as vectors, and how similarity between those vectors identifies relevant articles. Explain why titles may sometimes work better than full-article text for this search. Explain how the section sets up a DSPy retriever to find article titles similar to a question. Describe the roles of the embedder, the title corpus, and k. Trace how retrieval results are used to obtain Wix knowledge-base article text.

What does section 9.1, The WixQA dataset, cover?

The section introduces WixQA’s expert-written answers and knowledge base articles as the data for a RAG-based chatbot. Explain that WixQA provides Wix customer questions, documents that support answers, and expert-written answers assumed to be correct, which a RAG application aims to replicate. Describe the two WixQA subsets used here and how article IDs in a question-and-answer row identify supporting knowledge base entries, sometimes more than one.

What does section 9.2, Creating a DSPy RAG application, cover?

Introduces a DSPy RAG application that retrieves relevant WixQA articles by vector similarity and supplies their content to an LM to answer a question. Explain how a DSPy RAG module first gathers relevant data and then passes that data with the original query to an LM for an answer. Describe how article titles and a customer question are represented as vectors, and how similarity between those vectors identifies relevant articles. Explain why titles may sometimes work better than full-article text for this search. Explain how the content of the most similar articles becomes context in the LM prompt, and why retrieval should return enough articles to answer without supplying too many.

What does section 9.2.1, Embedding the article titles, cover?

The section shows how to use embeddings to retrieve relevant Wix knowledge-base article titles and then locate the corresponding article text. Explain how the section sets up a DSPy retriever to find article titles similar to a question. Describe the roles of the embedder, the title corpus, and k. Explain why DSPy keeps this title corpus in memory rather than building a Faiss database. Trace how retrieval results are used to obtain Wix knowledge-base article text.

What does section 9.2.2, The baseline RAG module, cover?

The section explains how a baseline DSPy RAG module retrieves relevant knowledge-base articles and uses an LM to answer a question. Trace how the module answers a question: its forward() method retrieves the k most similar articles, then passes those articles and the original question to the LM to generate a response. Explain that the RAG module follows the same custom-module approach as other DSPy modules and, once compiled into a DSPy program, can be evaluated and optimized with the same techniques.

What does section 9.2.3, The factuality evaluation metric, cover?

This section introduces a factuality judge for evaluating RAG answers against WixQA ground-truth answers and checks it on two known examples. Explain how WixQA’s expert-written answers make it possible to evaluate baseline RAG responses, and why this section uses LLM-as-a-judge rather than lexical or embedding comparison. Describe what the custom DSPy judge checks when scoring a predicted answer against the ground truth. Explain why the judge is first tried on two manually created examples and what their scores show.

What does section 9.2.4, Calculating the overall accuracy of the baseline, cover?

The section evaluates a baseline RAG program with a judge, compares two task LMs, and shows how to decide whether disappointing answers call for retrieval or answer-generation improvements. Explain how this baseline is evaluated using the judge directly rather than a separate metric function, and state what the two task-LM results showed. Distinguish missing necessary retrieved documents from incorrect answers generated despite appropriate documents, and explain which part of the RAG program each case suggests improving. Describe how comparing retrieved documents with golden documents can guide retrieval changes, including when to adjust k or consider other retrieval components.

What does section 9.2.5, Multihop RAG, cover?

This section explains how RAG can move beyond one retrieval step by searching iteratively, including a multistep module that uses a critic to decide when to stop. Trace how a multihop RAG workflow retrieves documents, has an LM identify missing information, and searches again to build a better answer. Describe how the example multistep module generates search terms, assesses answers with a critic, and decides when to stop. State how the advanced RAG module can be evaluated and optimized.

What does section 9.2.6, HyDE, cover?

Explains how HyDE improves RAG retrieval by generating hypothetical matching titles or document content, then contrasts RAG with fine-tuning a stable knowledge base into an LM. Explain why HyDE generates hypothetical titles or document content for retrieval instead of relying on the user's wording, and why it may generate several query strings. Distinguish fine-tuning an LM on a fairly stable local knowledge base from supplying retrieved context through RAG, including the stated benefit and cost of fine-tuning.

What does section 9.3, Agents and ReAct, cover?

Introduces DSPy’s ReAct agent as a reasoning-and-action loop that uses Python tools and their results to complete tasks. Trace how an agent reasons about a task, acts on a subtask, examines the result, and repeats until it finishes or reaches a step limit. Explain what ReAct tools are and distinguish the LM’s choice of tool and parameters from the module’s execution of the tool. Describe how returned values become observations that inform the next step. Describe how the task-level agent cycle can sit inside a longer-running loop that handles arriving user requests.

What does section 9.3.1, Creating a ReAct calculator, cover?

This section uses a four-operation DSPy calculator to show how ReAct selects and executes tools, records their results, and produces a final answer. Explain why the calculator uses ReAct and describe its inputs, output, tools, and iteration limit. Trace how the calculator moves from iterative tool selection and execution to a final LM-generated response. Describe what the trajectory records, how finish ends tool use, and what happens when max_iters is reached.

What does section 9.3.2, Agentic RAG, cover?

Explains how a ReAct-based RAG system chooses and repeatedly uses retrieval tools, illustrates that process with WixQA, and weighs its accuracy against its LM-call cost. Explain how agentic RAG uses an LM to choose and call data-collection tools, then reassesses what information it needs after retrieval. Trace the example's use of a title-retrieval tool and an article-reading tool before it answers. Justify when the section favors a simple, single-step RAG model over ReAct-based RAG, and describe the suggested response to useful information buried in long articles.

What does section 9.3.3, Agents using DSPy’s Python interpreter, cover?

This section explains why a ReAct agent might use Python execution and how DSPy’s PythonInterpreter serves as that tool. Explain why an agent may generate and execute Python code for a complex analytic problem or a simulation instead of relying on an LM’s direct answer. Describe how this example gives a ReAct agent Python execution through DSPy’s PythonInterpreter, and state what is required to run it.

What should I know before reading chapter 9?

Coding a custom module (chapter 7): Creating a custom DSPy module.; Evaluation using embeddings (chapter 8): Using embeddings to compare text.; Evaluation using LLM-as-a-judge (chapter 8): LLM-as-a-judge is an approach to evaluating generated answers.; Completeness (chapter 8): The section relates this factuality test particularly to completeness.; Modules (chapter 2): Introduces DSPy modules.; Signatures (chapter 2): Uses the idea that a signature specifies module inputs and outputs.; Refine (chapter 7): The earlier introduction to DSPy Refine.

Chapter 9: Creating an agentic RAG-based chatbot

What you will learn in chapter 9 of Building LLM Applications with DSPy

The introduction explains how retrieval, tool-using agents, and conversation history help DSPy chatbots answer questions and carry out requests.

After this chapter you will be able to

Retrieval for unfamiliar information. Explain how RAG lets a chatbot answer questions requiring information its language model did not see during training. Describe how it finds relevant documents and supplies them with the question as context for the model.
Language models and agent tools. Distinguish the language model's role in guiding tool use from the agent's role in executing tools to respond to a user's request.
WixQA reference data. Explain that WixQA provides Wix customer questions, documents that support answers, and expert-written answers assumed to be correct, which a RAG application aims to replicate.
Questions and knowledge base links. Describe the two WixQA subsets used here and how article IDs in a question-and-answer row identify supporting knowledge base entries, sometimes more than one.
RAG retrieval and generation. Explain how a DSPy RAG module first gathers relevant data and then passes that data with the original query to an LM for an answer.
Title embedding and vector search. Describe how article titles and a customer question are represented as vectors, and how similarity between those vectors identifies relevant articles. Explain why titles may sometimes work better than full-article text for this search.
Embedding-based title retrieval. Explain how the section sets up a DSPy retriever to find article titles similar to a question. Describe the roles of the embedder, the title corpus, and k.
Matched titles to article text. Trace how retrieval results are used to obtain Wix knowledge-base article text.
Baseline RAG response flow. Trace how the module answers a question: its forward() method retrieves the k most similar articles, then passes those articles and the original question to the LM to generate a response.
Ground-truth answer evaluation. Explain how WixQA’s expert-written answers make it possible to evaluate baseline RAG responses, and why this section uses LLM-as-a-judge rather than lexical or embedding comparison.
Factuality judge criterion. Describe what the custom DSPy judge checks when scoring a predicted answer against the ground truth.
Direct judge evaluation. Explain how this baseline is evaluated using the judge directly rather than a separate metric function, and state what the two task-LM results showed.
Two sources of RAG errors. Distinguish missing necessary retrieved documents from incorrect answers generated despite appropriate documents, and explain which part of the RAG program each case suggests improving.
Golden document retrieval checks. Describe how comparing retrieved documents with golden documents can guide retrieval changes, including when to adjust k or consider other retrieval components.
Multihop retrieval workflow. Trace how a multihop RAG workflow retrieves documents, has an LM identify missing information, and searches again to build a better answer.
Critic-guided multistep loop. Describe how the example multistep module generates search terms, assesses answers with a critic, and decides when to stop.
HyDE retrieval queries. Explain why HyDE generates hypothetical titles or document content for retrieval instead of relying on the user's wording, and why it may generate several query strings.
ReAct reasoning and action cycle. Trace how an agent reasons about a task, acts on a subtask, examines the result, and repeats until it finishes or reaches a step limit.
Tool execution and observations. Explain what ReAct tools are and distinguish the LM’s choice of tool and parameters from the module’s execution of the tool. Describe how returned values become observations that inform the next step.
Two-stage ReAct execution. Trace how the calculator moves from iterative tool selection and execution to a final LM-generated response.
Trajectory and stopping conditions. Describe what the trajectory records, how finish ends tool use, and what happens when max_iters is reached.
Iterative tool-guided retrieval. Explain how agentic RAG uses an LM to choose and call data-collection tools, then reassesses what information it needs after retrieval.
Accuracy and call-cost tradeoff. Justify when the section favors a simple, single-step RAG model over ReAct-based RAG, and describe the suggested response to useful information buried in long articles.
Code-based problem solving. Explain why an agent may generate and execute Python code for a complex analytic problem or a simulation instead of relying on an LM’s direct answer.
Python execution as a tool. Describe how this example gives a ReAct agent Python execution through DSPy’s PythonInterpreter, and state what is required to run it.
Memory for follow-up conversations. Explain why a service chatbot needs a history of interactions to handle questions that depend on earlier exchanges.
History in language model prompts. Describe how this section supplies memory to the LM and why information in that history may need to take precedence over general knowledge.
Refine response selection. Trace how Refine runs the ReAct submodule up to the specified N, uses a reward function and threshold to identify a satisfactory response, and otherwise returns the best response.
Repeated tool execution safety. Explain why placing ReAct inside Refine requires attention to tool safety, and distinguish harmless repeated lookups from tool executions that may need caching or limits.
MCP tool interaction. Describe how an MCP server and client let an AI application discover and invoke externally provided tools.
MCP host client server roles. Describe the host as the user-facing application that manages one or more MCP clients, and explain that each client interacts with an MCP server.
MCP connection types. Distinguish stdio from HTTP connections by where the communicating processes or machines are located.
Reusable knowledge base tools. Explain why Wix exposes its knowledge-base operations through an MCP server and describe what its two tools return.
Discoverable tool definitions. Describe how FastMCP registers Python functions as discoverable tools and why their descriptions and parameter definitions matter to an agent.
MCP client setup and discovery. Trace how the DSPy client starts the MCP server, initializes a session, and discovers available tools without importing their implementations.
MCP tool conversion and execution. Explain how discovered MCP tools become usable by DSPy ReAct and where they execute.
Agent control and asynchronous calls. Distinguish the server's role from the agent's tool-selection role and explain why this agent is called asynchronously.

Cover of Building LLM Applications with DSPy

liveBook manning.com

Read chapter 9 on liveBook Get the book at manning.com

What it builds on

Coding a custom module (chapter 7) · Creating a custom DSPy module.
Evaluation using embeddings (chapter 8) · Using embeddings to compare text.
Evaluation using LLM-as-a-judge (chapter 8) · LLM-as-a-judge is an approach to evaluating generated answers.
Completeness (chapter 8) · The section relates this factuality test particularly to completeness.
Modules (chapter 2) · Introduces DSPy modules.
Signatures (chapter 2) · Uses the idea that a signature specifies module inputs and outputs.
Refine (chapter 7) · The earlier introduction to DSPy Refine.

Section by section

Introduction

The introduction explains how retrieval, tool-using agents, and conversation history help DSPy chatbots answer questions and carry out requests.

Retrieval for unfamiliar information: Explain how RAG lets a chatbot answer questions requiring information its language model did not see during training. Describe how it finds relevant documents and supplies them with the question as context for the model.
Why it matters: Retrieval gives the model the company information it needs to answer a customer’s question.
Language models and agent tools: Distinguish the language model's role in guiding tool use from the agent's role in executing tools to respond to a user's request.
Why it matters: This distinction explains how a chatbot can do more than generate a written answer.
Conversation history for context: Explain why maintaining discussion history helps a chatbot answer questions and use tools in light of earlier conversation.
Why it matters: Remembering earlier discussion helps a chatbot respond in context rather than treating every request as isolated.

Key terms: retrieval-augmented generation, RAG, knowledge base, contextual information, agentic systems, agents, LMs, tools, history of discussions, memory

9.1 The WixQA dataset

The section introduces WixQA’s expert-written answers and knowledge base articles as the data for a RAG-based chatbot.

WixQA reference data: Explain that WixQA provides Wix customer questions, documents that support answers, and expert-written answers assumed to be correct, which a RAG application aims to replicate.
Why it matters: This explains what the chatbot is trying to answer and what counts as a reference answer.
Questions and knowledge base links: Describe the two WixQA subsets used here and how article IDs in a question-and-answer row identify supporting knowledge base entries, sometimes more than one.
Why it matters: The article IDs show how questions and answers connect to the knowledge base used for retrieval.

Key terms: WixQA, queries, golden documents, golden answers, wixqa_expertwritten, wix_kb_ corpus, article IDs, knowledge base

9.2 Creating a DSPy RAG application

Introduces a DSPy RAG application that retrieves relevant WixQA articles by vector similarity and supplies their content to an LM to answer a question.

RAG retrieval and generation: Explain how a DSPy RAG module first gathers relevant data and then passes that data with the original query to an LM for an answer.
Why it matters: This division explains what retrieval adds to an ordinary LM prompt.
Title embedding and vector search: Describe how article titles and a customer question are represented as vectors, and how similarity between those vectors identifies relevant articles. Explain why titles may sometimes work better than full-article text for this search.
Why it matters: It explains how the application chooses which knowledge-base articles might help answer a question.
Retrieved content and selection balance: Explain how the content of the most similar articles becomes context in the LM prompt, and why retrieval should return enough articles to answer without supplying too many.
Why it matters: Finding relevant titles helps only if the LM receives useful article content in a manageable amount.

Key terms: RAG, retrieval, generation, custom module, vector search, embedding, article titles, vector store, context, vector DB

9.2.1 Embedding the article titles

The section shows how to use embeddings to retrieve relevant Wix knowledge-base article titles and then locate the corresponding article text.

Embedding-based title retrieval: Explain how the section sets up a DSPy retriever to find article titles similar to a question. Describe the roles of the embedder, the title corpus, and k.
Why it matters: Retrieving relevant titles is how this application identifies articles that may help answer a question.
Corpus-size storage choice: Explain why DSPy keeps this title corpus in memory rather than building a Faiss database.
Why it matters: It explains the storage behavior of the retriever used for this relatively small knowledge base.
Matched titles to article text: Trace how retrieval results are used to obtain Wix knowledge-base article text.
Why it matters: The retrieved title matches must lead back to article text before they can support the RAG application.

Key terms: SentenceTransformer, bge-small-en-v1.5, DSPy retriever, Embeddings, corpus, k, vector store, Faiss, in memory, WixQA

9.2.2 The baseline RAG module

The section explains how a baseline DSPy RAG module retrieves relevant knowledge-base articles and uses an LM to answer a question.

Baseline RAG response flow: Trace how the module answers a question: its forward() method retrieves the k most similar articles, then passes those articles and the original question to the LM to generate a response.
Why it matters: This is the basic retrieval-and-generation flow used to answer questions from the knowledge base.
RAG as a DSPy module: Explain that the RAG module follows the same custom-module approach as other DSPy modules and, once compiled into a DSPy program, can be evaluated and optimized with the same techniques.
Why it matters: The familiar DSPy workflow still applies when the module includes retrieval.

Key terms: RAG module, forward(), retrieve step, generation step, k most similar articles, custom modules, DSPy program, evaluate, optimize

9.2.3 The factuality evaluation metric

This section introduces a factuality judge for evaluating RAG answers against WixQA ground-truth answers and checks it on two known examples.

Ground-truth answer evaluation: Explain how WixQA’s expert-written answers make it possible to evaluate baseline RAG responses, and why this section uses LLM-as-a-judge rather than lexical or embedding comparison.
Why it matters: The expert answers provide a reference for judging whether the chatbot’s answers are good.
Factuality judge criterion: Describe what the custom DSPy judge checks when scoring a predicted answer against the ground truth.
Why it matters: This criterion gives meaning to the score rather than treating it as an unexplained number.
Known-answer judge check: Explain why the judge is first tried on two manually created examples and what their scores show.
Why it matters: Known examples provide a quick sanity check before applying the judge to RAG responses.

Key terms: ground truth answer, golden answers, LLM-as-a-judge, factuality, prediction, ground truth, completeness, judge, baseline RAG module

9.2.4 Calculating the overall accuracy of the baseline

The section evaluates a baseline RAG program with a judge, compares two task LMs, and shows how to decide whether disappointing answers call for retrieval or answer-generation improvements.

Direct judge evaluation: Explain how this baseline is evaluated using the judge directly rather than a separate metric function, and state what the two task-LM results showed.
Why it matters: This gives you a way to evaluate the baseline and interpret its task-LM comparison.
Two sources of RAG errors: Distinguish missing necessary retrieved documents from incorrect answers generated despite appropriate documents, and explain which part of the RAG program each case suggests improving.
Why it matters: Separating retrieval failures from answer-generation failures helps you improve the right part of the program.
Golden document retrieval checks: Describe how comparing retrieved documents with golden documents can guide retrieval changes, including when to adjust k or consider other retrieval components.
Why it matters: A document-level check makes retrieval changes more targeted than relying on the overall answer score alone.

Key terms: judge, metric function, forward(), task LM, example, prediction, retriever, LM, prompt, RAG program

9.2.5 Multihop RAG

This section explains how RAG can move beyond one retrieval step by searching iteratively, including a multistep module that uses a critic to decide when to stop.

Multihop retrieval workflow: Trace how a multihop RAG workflow retrieves documents, has an LM identify missing information, and searches again to build a better answer.
Why it matters: Repeated retrieval lets the application pursue information that an initial set of articles did not provide.
Critic-guided multistep loop: Describe how the example multistep module generates search terms, assesses answers with a critic, and decides when to stop.
Why it matters: The critic gives this repeated-search workflow a way to assess its answer and a stopping condition.
Advanced RAG evaluation: State how the advanced RAG module can be evaluated and optimized.
Why it matters: A more elaborate retrieval workflow can still be assessed and improved using the DSPy approach.

Key terms: baseline RAG, multihop workflow, retriever, LM, multistep workflow, search terms, critic, maximum number of steps, RAG module, evaluated

9.2.6 HyDE

Explains how HyDE improves RAG retrieval by generating hypothetical matching titles or document content, then contrasts RAG with fine-tuning a stable knowledge base into an LM.

HyDE retrieval queries: Explain why HyDE generates hypothetical titles or document content for retrieval instead of relying on the user's wording, and why it may generate several query strings.
Why it matters: It explains how retrieval can find relevant material even when a user's question and an article title use different words.
Fine-tuning versus retrieved context: Distinguish fine-tuning an LM on a fairly stable local knowledge base from supplying retrieved context through RAG, including the stated benefit and cost of fine-tuning.
Why it matters: It clarifies when training knowledge into a model might replace repeated retrieval and why RAG can remain the more practical choice.

Key terms: Hypothetical Document Embedding (HyDE), retriever, hypothetical query strings, fine-tune, model’s weights, local knowledge base, context information

9.3 Agents and ReAct

Introduces DSPy’s ReAct agent as a reasoning-and-action loop that uses Python tools and their results to complete tasks.

ReAct reasoning and action cycle: Trace how an agent reasons about a task, acts on a subtask, examines the result, and repeats until it finishes or reaches a step limit.
Why it matters: This cycle explains how an agent can work through a task in steps.
Tool execution and observations: Explain what ReAct tools are and distinguish the LM’s choice of tool and parameters from the module’s execution of the tool. Describe how returned values become observations that inform the next step.
Why it matters: Separating tool selection from execution makes the agent’s actions and feedback easier to explain.
Long-running assistant loop: Describe how the task-level agent cycle can sit inside a longer-running loop that handles arriving user requests.
Why it matters: The outer loop explains how a task-solving agent can serve successive requests.

Key terms: ReAct, reasoning, acting, subtasks, ReAct module, tools, Python functions, observation, long-running loop, always-ready assistant

9.3.1 Creating a ReAct calculator

This section uses a four-operation DSPy calculator to show how ReAct selects and executes tools, records their results, and produces a final answer.

Calculator tools and signature: Explain why the calculator uses ReAct and describe its inputs, output, tools, and iteration limit.
Why it matters: The calculator makes the division of work between the LM and its tools concrete.
Two-stage ReAct execution: Trace how the calculator moves from iterative tool selection and execution to a final LM-generated response.
Why it matters: Separating tool use from final response generation explains what the agent does at each stage.
Trajectory and stopping conditions: Describe what the trajectory records, how finish ends tool use, and what happens when max_iters is reached.
Why it matters: The trajectory shows what the agent has done, while the stopping limit can affect answer quality.
Tool selection prompt details: Explain what DSPy supplies to the LM for tool selection and why clear tool definitions matter.
Why it matters: Clear tool descriptions help the LM choose and supply arguments to the intended function.

Key terms: ReAct, signature, tools, max_iters, reasoning–acting loop, tool executions, observations, final response, trajectory, finish

9.3.2 Agentic RAG

Explains how a ReAct-based RAG system chooses and repeatedly uses retrieval tools, illustrates that process with WixQA, and weighs its accuracy against its LM-call cost.

Iterative tool-guided retrieval: Explain how agentic RAG uses an LM to choose and call data-collection tools, then reassesses what information it needs after retrieval.
Why it matters: This is what makes retrieval responsive to information found along the way rather than a single fixed lookup.
WixQA tool-call sequence: Trace the example's use of a title-retrieval tool and an article-reading tool before it answers.
Why it matters: The sequence makes the iterative tool use concrete.
Accuracy and call-cost tradeoff: Justify when the section favors a simple, single-step RAG model over ReAct-based RAG, and describe the suggested response to useful information buried in long articles.
Why it matters: The choice of approach depends on whether the potential accuracy gain warrants extra LM calls.

Key terms: agentic RAG, ReAct, tools, multistep process, WixQA, retrieve_article_titles_and_ids(), read_article(), SupportBackend, ReAct-based approach, single-step model

9.3.3 Agents using DSPy’s Python interpreter

This section explains why a ReAct agent might use Python execution and how DSPy’s PythonInterpreter serves as that tool.

Code-based problem solving: Explain why an agent may generate and execute Python code for a complex analytic problem or a simulation instead of relying on an LM’s direct answer.
Why it matters: It explains what the code-execution tool adds to the agent.
Python execution as a tool: Describe how this example gives a ReAct agent Python execution through DSPy’s PythonInterpreter, and state what is required to run it.
Why it matters: It shows how generated code becomes an executable tool call in this agent example.

Key terms: Python code, complex analytic solution, simulation, ReAct agent, PythonInterpreter, Deno, Pyodide

9.4 Maintaining memory

This section explains how passing conversation history to a DSPy chatbot helps it answer follow-up questions, and what a production memory implementation must account for.

Memory for follow-up conversations: Explain why a service chatbot needs a history of interactions to handle questions that depend on earlier exchanges.
Why it matters: Remembering prior turns lets the chatbot respond to a continuing conversation rather than treating every question as isolated.
History in language model prompts: Describe how this section supplies memory to the LM and why information in that history may need to take precedence over general knowledge.
Why it matters: Including history in the prompt gives the LM context that its general knowledge alone cannot supply.
Production memory boundaries: State the two improvements this section identifies for using chatbot memory in production.
Why it matters: A working demonstration of memory still needs limits and per-user separation before it is suitable for a service chatbot.

Key terms: memory, history of previous interactions, DSPy’s History class, history, prompts, general knowledge, relevant items in history, separate histories per user, bloat

9.5 Self-correction with DSPy Refine

This section shows how DSPy Refine can improve an inconsistent chatbot by trying responses against a quality check, while warning that repeated attempts may rerun tools.

Refine quality criteria: Explain when Refine is useful for a chatbot and what its reward function can check. Describe the section’s 100-word limit as a simple example rather than the only possible quality criterion.
Why it matters: A quality check tells Refine what counts as a better chatbot response.
Refine response selection: Trace how Refine runs the ReAct submodule up to the specified N, uses a reward function and threshold to identify a satisfactory response, and otherwise returns the best response.
Why it matters: The retry limit, quality threshold, and fallback explain which answer the chatbot ultimately returns.
Repeated tool execution safety: Explain why placing ReAct inside Refine requires attention to tool safety, and distinguish harmless repeated lookups from tool executions that may need caching or limits.
Why it matters: An extra attempt at an answer should not accidentally repeat an unsafe tool action.

Key terms: Refine, reward function, LLM-as-a-judge, ReAct submodule, threshold, N, ReAct, tools, cache

9.6 Connecting agents to MCP

This section explains why agents need a standard way to use external capabilities and introduces MCP’s server-and-client approach to tool discovery and invocation.

Limits of local tools: Explain why tools defined as Python functions alongside an agent may be insufficient in larger systems, and why using services through APIs ordinarily requires developer-written tool wrappers.
Why it matters: This explains the integration problem that MCP is intended to address.
MCP tool interaction: Describe how an MCP server and client let an AI application discover and invoke externally provided tools.
Why it matters: The server–client division is the basic model for connecting an agent to MCP tools.
MCP and existing APIs: Distinguish MCP from the underlying APIs it may use, and explain why multiple AI clients can benefit from one MCP server.
Why it matters: A shared MCP interface can reduce repeated integrations without requiring existing services to be replaced.

Key terms: Python functions, API, SDK, tool, Model Context Protocol (MCP), MCP server, MCP client, tools, input schemas, database

9.6.1 MCP protocol intro

Introduces the MCP host, client, and server roles and the two connection types needed to understand how an application reaches MCP services.

MCP host client server roles: Describe the host as the user-facing application that manages one or more MCP clients, and explain that each client interacts with an MCP server.
Why it matters: These roles clarify where the application sits and how it reaches MCP services.
MCP connection types: Distinguish stdio from HTTP connections by where the communicating processes or machines are located.
Why it matters: The connection type tells you whether the MCP server is reached locally or on a remote machine.

Key terms: host, MCP client, MCP server, stdio, standard input/output, HTTP

9.6.2 Exposing the Wix knowledge base through an MCP server

The section shows how to expose WixQA knowledge-base search and article reading through a FastMCP server for other AI applications.

Reusable knowledge base tools: Explain why Wix exposes its knowledge-base operations through an MCP server and describe what its two tools return.
Why it matters: This separates reusable knowledge-base access from each application that needs it.
Discoverable tool definitions: Describe how FastMCP registers Python functions as discoverable tools and why their descriptions and parameter definitions matter to an agent.
Why it matters: A clear tool interface helps the agent select the intended operation and call it correctly.
Thin knowledge base wrappers: Describe how the server's tool functions relate to the existing WixQA backend.
Why it matters: The MCP interface can reuse the existing retrieval implementation.
Server startup and transport: Explain how the example starts its MCP server and how its client communicates with it.
Why it matters: The client must use the transport on which this example's server is running.

Key terms: MCP server, search_wix_kb, read_wix_kb_article, FastMCP, @mcp.tool(), tool definition, docstring, type hints, knowledge base backend, WixQA articles

9.6.3 Connecting a DSPy agent to an MCP server

This section explains how a DSPy ReAct agent connects to an MCP server, discovers and uses its tools, and calls them asynchronously.

MCP client setup and discovery: Trace how the DSPy client starts the MCP server, initializes a session, and discovers available tools without importing their implementations.
Why it matters: Runtime discovery lets the agent use server-published tools without importing their Python implementations.
MCP tool conversion and execution: Explain how discovered MCP tools become usable by DSPy ReAct and where they execute.
Why it matters: The agent can use remotely executed MCP tools through the same DSPy tool interface as local tools.
Agent control and asynchronous calls: Distinguish the server's role from the agent's tool-selection role and explain why this agent is called asynchronously.
Why it matters: Connecting tools through MCP changes how calls are made, not who chooses the tools or how ReAct uses their results.

Key terms: StdioServerParameters, initialize(), list_tools(), MCP session, dspy.Tool.from_mcp_tool(), dspy.Tool, client session, observation, MCP host, MCP server

9.7 Summary

Concepts explained in this chapter

ReActReAct is a DSPy module that implements an agent loop in which a language model selects tools and uses their results to guide subsequent steps.
knowledge baseThe knowledge base is the example application's collection of Wix knowledge-base articles used as supporting information for answers.
MCP serverA component that exposes tools or data to an MCP client through the Model Context Protocol.
retrieval-augmented generationA language-model application pattern that retrieves external information and supplies it as context for response generation.
ReActThe ReAct pattern guides an agent to interleave reasoning with actions such as selecting and invoking tools.
mcp clientA protocol component that connects to MCP servers to discover and invoke their capabilities.
vector databaseA vector store is a data system that indexes and retrieves numerical vector representations, typically by similarity.
MCP hostThe MCP host is the application that manages an MCP connection and decides when to invoke tools exposed by an MCP server.
moduleA DSPy module composed within another module and invoked as part of its computation.
toolAn invocable software capability that performs an operation on behalf of an application or agent.
embeddingA numerical vector representation of an item, such as text, that encodes useful features.
MCPModel Context Protocol is a standard protocol for connecting language-model applications with external tools and data sources.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.