What will I learn in chapter 7 of AI Agents and Applications?

Introduces the chapter’s planned coverage of building RAG chatbots with LangChain and tracing their execution with LangSmith. After reading it you should be able to: Explain how LangChain connects components such as data loaders, vector stores, and LLMs while allowing a provider to be replaced without redesigning the application. Trace how a loader turns source text into a Document, a TextSplitter divides it into chunks, an Embeddings model creates vector representations, and a VectorStore stores the chunks and embeddings. Trace how the Retriever embeds a question, searches the VectorStore for text with similar embeddings, and returns relevant chunks to the Q&A orchestrator. Describe how the Q&A orchestrator combines the user’s question with retrieved documents in a PromptTemplate and sends the resulting prompt to the LanguageModel for an answer. Explain why storing documents in a vector database comes before querying them in this chapter’s Q&A workflow. Describe the ingestion sequence: load each document, split it into roughly 500-character chunks with no overlap in this example, calculate chunk embeddings, and store the chunks with their embeddings in Chroma. Explain why the repeated code is extracted into a function and when that function is called. Describe how to ingest a folder by iterating over its files, identifying each file type, selecting the relevant loader, and importing the loaded content into the vector store.

What does section 7.1, LangChain object model for Q&A chatbots, cover?

This section explains how LangChain’s abstractions and ready-made components support Q&A applications, then introduces the two stages discussed next. Explain how LangChain connects components such as data loaders, vector stores, and LLMs while allowing a provider to be replaced without redesigning the application. Describe the kinds of common LLM-development tasks for which LangChain supplies ready-to-use implementations.

What does section 7.1.1, Content ingestion (indexing) stage, cover?

This section explains the LangChain components and sequence used to ingest content into a vector store for Q&A. Trace how a loader turns source text into a Document, a TextSplitter divides it into chunks, an Embeddings model creates vector representations, and a VectorStore stores the chunks and embeddings. Explain when separate embedding computation can be skipped during ingestion.

What does section 7.1.2, Q&A (retrieval and generation) stage, cover?

This section explains the LangChain components and sequence that retrieve relevant document chunks and use them to answer a question. Trace how the Retriever embeds a question, searches the VectorStore for text with similar embeddings, and returns relevant chunks to the Q&A orchestrator. Describe how the Q&A orchestrator combines the user’s question with retrieved documents in a PromptTemplate and sends the resulting prompt to the LanguageModel for an answer.

What does section 7.2, Vector store content ingestion, cover?

This section establishes that documents must be stored before they can be queried and sets up the environment for ingesting them. Explain why storing documents in a vector database comes before querying them in this chapter’s Q&A workflow. Describe the preparation steps given here: install packages for loading, splitting, and embedding documents in the chapter 7 virtual environment, then start a notebook, import the libraries, and capture the OpenAI API key.

What does section 7.2.1, Splitting and storing the documents, cover?

This section shows how to make documents searchable by splitting them into chunks and storing their embeddings in Chroma, while noting that Wikipedia loading may bring in extra articles. Describe the ingestion sequence: load each document, split it into roughly 500-character chunks with no overlap in this example, calculate chunk embeddings, and store the chunks with their embeddings in Chroma. Explain why loading a requested Wikipedia article may produce more content than expected.

What does section 7.2.2, Removing duplication, cover?

This section removes duplication in content-ingestion code by extracting shared work into a function called for each loader. Explain why the repeated code is extracted into a function and when that function is called.

What does section 7.2.3, Ingesting multiple documents from a folder, cover?

This section presents two ways to ingest a folder of documents into a vector store: selecting a loader for each file or using DirectoryLoader. Describe how to ingest a folder by iterating over its files, identifying each file type, selecting the relevant loader, and importing the loaded content into the vector store. Explain how DirectoryLoader offers an alternative for loading folder contents, what its folder path and glob pattern specify, and what installation caveat applies.

What does section 7.3, Q&A across stored documents, cover?

The section explains why retrieving information across stored documents is a useful check on content ingestion and chunking. Explain why asking a question that spans multiple sources helps check the ingestion and chunking strategy: the vector store should surface the most relevant pieces of stored text.

What does section 7.3.1, Querying the vector store directly, cover?

A direct vector-store query shows which chunks are retrieved for a question spanning sources, before a LangChain chain produces a formulated answer. Explain that querying the vector store directly reveals the relevant document chunks for a question, including chunks from different sources, and that their metadata shows where they came from.

What does section 7.3.2, Asking a question through a LangChain chain, cover?

This subsection explains how a LangChain RAG chain handles prompt setup and presents two ways to obtain its prompt from the LangChain Hub. Explain how a LangChain RAG chain differs from manually preparing a prompt with the question and vector-store context. Describe the two prompt setup options given for a template from the LangChain Hub.

What does section 7.3.3, Completing the RAG chain setup, cover?

This section completes a RAG chain by connecting retrieved context and the user’s question to a prompt that a chat model uses to generate an answer. Describe how the retriever, question feeder, and chat model contribute to answering a question in the completed RAG chain. Explain why the first block of this LangChain chain is a dictionary.

What should I know before reading chapter 7?

LangChain’s core object model (chapter 1): LangChain’s core object model; Storing text and performing a semantic search using Chroma (chapter 6): Introduces storing text and searching with a vector store.; Retrieving content from the vector database (chapter 6): Retrieving content from a vector database.; Invoking the LLM (chapter 6): Invoking an LLM to generate an answer.; Implementing RAG from scratch (chapter 6): The earlier implementation of RAG provides context for retrieving stored content.

Chapter 7: Q&A chatbots with LangChain and LangSmith

What you will learn in chapter 7 of AI Agents and Applications

Introduces the chapter’s planned coverage of building RAG chatbots with LangChain and tracing their execution with LangSmith.

After this chapter you will be able to

Interchangeable application components. Explain how LangChain connects components such as data loaders, vector stores, and LLMs while allowing a provider to be replaced without redesigning the application.
Content ingestion flow. Trace how a loader turns source text into a Document, a TextSplitter divides it into chunks, an Embeddings model creates vector representations, and a VectorStore stores the chunks and embeddings.
Embedding-Based Document Retrieval. Trace how the Retriever embeds a question, searches the VectorStore for text with similar embeddings, and returns relevant chunks to the Q&A orchestrator.
Prompt Construction and Answer Generation. Describe how the Q&A orchestrator combines the user’s question with retrieved documents in a PromptTemplate and sends the resulting prompt to the LanguageModel for an answer.
Storage before document queries. Explain why storing documents in a vector database comes before querying them in this chapter’s Q&A workflow.
Chunking and vector storage. Describe the ingestion sequence: load each document, split it into roughly 500-character chunks with no overlap in this example, calculate chunk embeddings, and store the chunks with their embeddings in Chroma.
Shared ingestion function. Explain why the repeated code is extracted into a function and when that function is called.
File-by-file loader selection. Describe how to ingest a folder by iterating over its files, identifying each file type, selecting the relevant loader, and importing the loaded content into the vector store.
DirectoryLoader folder ingestion. Explain how DirectoryLoader offers an alternative for loading folder contents, what its folder path and glob pattern specify, and what installation caveat applies.
Cross-document retrieval as a check. Explain why asking a question that spans multiple sources helps check the ingestion and chunking strategy: the vector store should surface the most relevant pieces of stored text.
Direct Query Retrieval Results. Explain that querying the vector store directly reveals the relevant document chunks for a question, including chunks from different sources, and that their metadata shows where they came from.
Automatic RAG prompt handling. Explain how a LangChain RAG chain differs from manually preparing a prompt with the question and vector-store context.
RAG chain component roles. Describe how the retriever, question feeder, and chat model contribute to answering a question in the completed RAG chain.
Stateless Follow-up Question Failure. Explain why the chatbot fails to understand that “they” refers to the Romans in a follow-up question.
Message history for follow-ups. Explain how remembering previous questions and responses helps an LLM-based chatbot support follow-up questions, unlike an LLM-based engine without that conversational context.
Message-based RAG prompt. Explain why the original RAG prompt must change for a memory-enabled chatbot and why ChatPromptTemplate.from_messages is suitable.
Prompt inputs across interactions. Trace which inputs are supplied to the message-based prompt at the initial interaction and on later interactions.
Messages Worth Tracking. Explain how ChatMessageHistory stores the conversation and why it tracks human and AI messages rather than system and chat_history messages.
History for the Next Question. Describe how the updated chat history is retrieved and used when the user asks another question.
Chat history chain input. Explain that after message history is added to the prompt and updated after each exchange, the RAG chain must pass it through the corresponding chat_history_messages property.
Memory-enabled RAG workflow. Describe how the assembled chain retrieves chat history, inserts it into the prompt, and updates it after each interaction, with LCEL connecting the retriever, prompt, memory, and model.
Contextual follow-up response. Explain what the test shows when the chatbot answers a follow-up question using its accumulated chat history.
Q&A tracing setup. Describe how to enable LangSmith tracing for the Q&A chatbot notebook, from creating an API key through rerunning the notebook.
Trace and run inspection. Trace how to inspect the Q&A chatbot's execution in LangSmith, distinguishing a project's traces from the component runs inside a selected trace.

liveBook manning.com

Read chapter 7 on liveBook Get the book at manning.com

What it builds on

LangChain’s core object model (chapter 1) · LangChain’s core object model
Storing text and performing a semantic search using Chroma (chapter 6) · Introduces storing text and searching with a vector store.
Retrieving content from the vector database (chapter 6) · Retrieving content from a vector database.
Invoking the LLM (chapter 6) · Invoking an LLM to generate an answer.
Implementing RAG from scratch (chapter 6) · The earlier implementation of RAG provides context for retrieving stored content.

Section by section

Introduction

Introduces the chapter’s planned coverage of building RAG chatbots with LangChain and tracing their execution with LangSmith.

7.1 LangChain object model for Q&A chatbots

This section explains how LangChain’s abstractions and ready-made components support Q&A applications, then introduces the two stages discussed next.

Interchangeable application components: Explain how LangChain connects components such as data loaders, vector stores, and LLMs while allowing a provider to be replaced without redesigning the application.
Why it matters: This explains why a Q&A application can change providers without changing its overall design.
Ready-made development tasks: Describe the kinds of common LLM-development tasks for which LangChain supplies ready-to-use implementations.
Why it matters: These built-in tasks reduce how much common Q&A application plumbing you must implement yourself.

Key terms: abstractions, data loaders, vector stores, LLMs, ready-to-use implementations, splitting source text, retrieving relevant context, generating prompts, managing the context window

7.1.1 Content ingestion (indexing) stage

This section explains the LangChain components and sequence used to ingest content into a vector store for Q&A.

Content ingestion flow: Trace how a loader turns source text into a Document, a TextSplitter divides it into chunks, an Embeddings model creates vector representations, and a VectorStore stores the chunks and embeddings.
Why it matters: This flow shows how source content becomes stored material that a Q&A chatbot can retrieve.
Automatic embedding during storage: Explain when separate embedding computation can be skipped during ingestion.
Why it matters: It explains why ingestion code may add chunks without an explicit earlier embedding step.

Key terms: Loader, Document, DocumentTransformer, TextSplitter, Embeddings, VectorStore, vector stores, embedding model, precompute the embeddings

7.1.2 Q&A (retrieval and generation) stage

This section explains the LangChain components and sequence that retrieve relevant document chunks and use them to answer a question.

Embedding-Based Document Retrieval: Trace how the Retriever embeds a question, searches the VectorStore for text with similar embeddings, and returns relevant chunks to the Q&A orchestrator.
Why it matters: This explains how the chatbot finds document content relevant to a question before generating an answer.
Prompt Construction and Answer Generation: Describe how the Q&A orchestrator combines the user’s question with retrieved documents in a PromptTemplate and sends the resulting prompt to the LanguageModel for an answer.
Why it matters: Retrieval alone does not answer the question; the retrieved material must be supplied to the language model as context.

Key terms: Retriever, VectorStore, Embeddings, text chunks, Q&A orchestrator, PromptTemplate, LanguageModel, context

7.2 Vector store content ingestion

This section establishes that documents must be stored before they can be queried and sets up the environment for ingesting them.

Storage before document queries: Explain why storing documents in a vector database comes before querying them in this chapter’s Q&A workflow.
Why it matters: This prerequisite establishes the order of the ingestion and Q&A stages.
Ingestion workspace setup: Describe the preparation steps given here: install packages for loading, splitting, and embedding documents in the chapter 7 virtual environment, then start a notebook, import the libraries, and capture the OpenAI API key.
Why it matters: These setup steps prepare the workspace for the document-ingestion examples.

Key terms: vector database, querying your documents, virtual environment, Jupyter Notebook, OpenAI API key

7.2.1 Splitting and storing the documents

This section shows how to make documents searchable by splitting them into chunks and storing their embeddings in Chroma, while noting that Wikipedia loading may bring in extra articles.

Chunking and vector storage: Describe the ingestion sequence: load each document, split it into roughly 500-character chunks with no overlap in this example, calculate chunk embeddings, and store the chunks with their embeddings in Chroma.
Why it matters: This is the ingestion process that prepares documents for later searches and questions.
Wikipedia hyperlink loading: Explain why loading a requested Wikipedia article may produce more content than expected.
Why it matters: This explains why the loaded content may extend beyond the single article you requested.

Key terms: chunks, overlap, embeddings, Chroma vector database, WikipediaLoader, Wikipedia hyperlinks, related articles

7.2.2 Removing duplication

This section removes duplication in content-ingestion code by extracting shared work into a function called for each loader.

Shared ingestion function: Explain why the repeated code is extracted into a function and when that function is called.
Why it matters: One shared function avoids repeating the same ingestion work for each loader.

Key terms: duplication, function, loader

7.2.3 Ingesting multiple documents from a folder

This section presents two ways to ingest a folder of documents into a vector store: selecting a loader for each file or using DirectoryLoader.

File-by-file loader selection: Describe how to ingest a folder by iterating over its files, identifying each file type, selecting the relevant loader, and importing the loaded content into the vector store.
Why it matters: This approach lets you ingest many files even when they need different document loaders.
DirectoryLoader folder ingestion: Explain how DirectoryLoader offers an alternative for loading folder contents, what its folder path and glob pattern specify, and what installation caveat applies.
Why it matters: It provides a folder-level loading option, provided its third-party dependencies work in your environment.

Key terms: loader factory, get_loader(), split_and_import(), vector store, DirectoryLoader, folder path, glob pattern, Unstructured, langchain-unstructured

7.3 Q&A across stored documents

The section explains why retrieving information across stored documents is a useful check on content ingestion and chunking.

Cross-document retrieval as a check: Explain why asking a question that spans multiple sources helps check the ingestion and chunking strategy: the vector store should surface the most relevant pieces of stored text.
Why it matters: Relevant retrieval across sources is an early sign that the stored content is useful for answering questions.

Key terms: vector store, ingestion, chunking strategy

7.3.1 Querying the vector store directly

A direct vector-store query shows which chunks are retrieved for a question spanning sources, before a LangChain chain produces a formulated answer.

Direct Query Retrieval Results: Explain that querying the vector store directly reveals the relevant document chunks for a question, including chunks from different sources, and that their metadata shows where they came from.
Why it matters: It separates checking what was retrieved from generating an answer from that material.

Key terms: vector store, chunks, metadata, LangChain chain

7.3.2 Asking a question through a LangChain chain

This subsection explains how a LangChain RAG chain handles prompt setup and presents two ways to obtain its prompt from the LangChain Hub.

Automatic RAG prompt handling: Explain how a LangChain RAG chain differs from manually preparing a prompt with the question and vector-store context.
Why it matters: This is the main simplification LangChain brings to the prompt setup described here.
LangChain Hub prompt options: Describe the two prompt setup options given for a template from the LangChain Hub.
Why it matters: Either option supplies the prompt for the LangChain setup without requiring a new prompt to be written from scratch.

Key terms: RAG chain, prompt template, vector store, LangChain Hub, prompt instance

7.3.3 Completing the RAG chain setup

This section completes a RAG chain by connecting retrieved context and the user’s question to a prompt that a chat model uses to generate an answer.

RAG chain component roles: Describe how the retriever, question feeder, and chat model contribute to answering a question in the completed RAG chain.
Why it matters: These roles explain how stored content and a user question reach the model in a RAG chain.
Dictionary input to chain: Explain why the first block of this LangChain chain is a dictionary.
Why it matters: The input format makes the chain’s initial setup easier to understand.

Key terms: Retriever, Question feeder, Chat model, context parameter, pass-through component, Runnable interface, dictionary, LangChain chain

7.3.4 Follow-up question

A vague follow-up question reveals that the chatbot cannot resolve references to an earlier exchange because it does not retain conversation history.

Stateless Follow-up Question Failure: Explain why the chatbot fails to understand that “they” refers to the Romans in a follow-up question.
Why it matters: A chatbot needs conversational context to make sense of follow-up questions that refer back to earlier turns.

Key terms: stateless, memory, conversation flow, follow-up question

7.4 Chatbot memory of message history

This section explains why remembering earlier questions and responses makes a chatbot useful for follow-up questions.

Message history for follow-ups: Explain how remembering previous questions and responses helps an LLM-based chatbot support follow-up questions, unlike an LLM-based engine without that conversational context.
Why it matters: Message history helps a chatbot respond to a question as part of an ongoing conversation.

Key terms: message history, previous questions and responses, context

7.4.1 Amending the prompt

This subsection adapts a RAG prompt to include chat history and explains the message format and inputs the revised prompt uses.

Message-based RAG prompt: Explain why the original RAG prompt must change for a memory-enabled chatbot and why ChatPromptTemplate.from_messages is suitable.
Why it matters: A message-based prompt gives the memory-enabled RAG design a way to include the conversation alongside the question and retrieved context.
Chat messages and history: Describe how the section represents an individual chat message and a chat history.
Why it matters: This representation explains what the prompt receives as chat history.
Prompt inputs across interactions: Trace which inputs are supplied to the message-based prompt at the initial interaction and on later interactions.
Why it matters: These inputs let each prompt reflect the current question and retrieved material together with the conversation so far.

Key terms: PromptTemplate.from_template, ChatPromptTemplate.from_messages, message history, chat message, role, message text, chat history, chat_history, question, retrieved_context

7.4.2 Updating the chat message history

This section shows how to keep user questions and LLM responses in LangChain chat history so they can be supplied with the next question.

Messages Worth Tracking: Explain how ChatMessageHistory stores the conversation and why it tracks human and AI messages rather than system and chat_history messages.
Why it matters: Tracking the conversation without duplicating prompt content gives the chatbot the relevant prior exchanges.
History for the Next Question: Describe how the updated chat history is retrieved and used when the user asks another question.
Why it matters: Retrieving and supplying the stored exchanges lets the next prompt include the conversation so far.

Key terms: ChatMessageHistory, chat_history_memory, human, ai, system, chat_history, messages property, prompt

7.4.3 Feeding the chat history to the RAG chain

This section connects updated message history to the RAG chain’s prompt using LCEL.

Chat history chain input: Explain that after message history is added to the prompt and updated after each exchange, the RAG chain must pass it through the corresponding chat_history_messages property.
Why it matters: Updating chat history is not enough unless the chain also supplies it to the prompt.

Key terms: message history, RAG chain, LangChain Expression Language (LCEL), chat_history_messages

7.4.4 Putting everything together

This section brings the memory-enabled RAG chain together and tests whether chat history lets it answer a contextual follow-up question.

Memory-enabled RAG workflow: Describe how the assembled chain retrieves chat history, inserts it into the prompt, and updates it after each interaction, with LCEL connecting the retriever, prompt, memory, and model.
Why it matters: This explains how earlier exchanges become available when the chatbot answers a new question.
Contextual follow-up response: Explain what the test shows when the chatbot answers a follow-up question using its accumulated chat history.
Why it matters: The test demonstrates why retaining conversation history matters for follow-up questions.

Key terms: chat history, LCEL, retriever, prompt, memory, model, follow-up question, memoryless chatbot

7.5 Tracing execution with LangSmith

The section explains why LangSmith is useful, how to enable tracing for the Q&A chatbot notebook, and how to inspect its execution in the dashboard.

LangSmith development lifecycle role: Explain why LangSmith helps with LLM applications and describe its roles in debugging, testing, and monitoring.
Why it matters: It explains why tracing an LLM application is useful rather than treating traces as merely a record of calls.
Q&A tracing setup: Describe how to enable LangSmith tracing for the Q&A chatbot notebook, from creating an API key through rerunning the notebook.
Why it matters: The dashboard can show the chatbot's execution only after tracing is configured and the notebook runs.
Trace and run inspection: Trace how to inspect the Q&A chatbot's execution in LangSmith, distinguishing a project's traces from the component runs inside a selected trace.
Why it matters: Inspecting individual runs lets you see what each chatbot component received and produced, not just the final answer.

Key terms: LangSmith, tracing, evaluators, monitoring, LangSmith API key, Personal Access Token, environment variables, Jupyter Notebook, project, trace

7.6 Summary

This section summarizes how LangChain assembles RAG chatbots, preserves conversation context, and uses LangSmith to inspect execution.

Where this is used later

Improving RAG accuracy (chapter 8) · Introduces the content ingestion and Q&A stages of a RAG chatbot.
Advanced document indexing (chapter 8) · Assumes a content ingestion and indexing stage.
Content ingestion stage (chapter 8) · The basic process of splitting and storing documents.
Question-answering stage (chapter 8) · Retrieving stored documents for question answering.
Summary (chapter 8) · Retrieving matches from a vector store
Similarity postprocessors (chapter 10) · Querying a vector store to retrieve documents.
Enabling LangSmith tracing (chapter 11) · Enabling LangSmith tracing for the later agent builds on this setup pattern.
Observing and debugging with LangSmith (chapter 11) · Observing and debugging a later agent with LangSmith uses the same idea of inspecting execution details.

Concepts explained in this chapter

dialogueA chat message history is an ordered collection of user and language-model messages representing a conversation.
trajectoryExecution history is the recorded sequence of workflow decisions, transitions, and intermediate states produced during graph execution.
LangSmithLangSmith is a development and observability platform for tracing, debugging, evaluating, monitoring, and refining language-model applications.
pipelineA multi-step workflow is a process composed of several ordered or coordinated operations that collectively accomplish a task.
vector databaseA vector store is a database specialized for storing embeddings and associated documents and retrieving them by semantic similarity.
LangSmith projectA LangSmith project is a collection of execution traces associated with an application or part of an application.
evaluatorAn evaluator is a model or component that assesses outputs against specified criteria and produces judgments, scores, comparisons, or feedback.
embeddingA vector type is a database data type that represents numerical vector values as stored fields.
vectorizationAn embedding model is a model that maps text or other data to numerical vectors representing semantic or contextual properties.
text chunkA text splitter is a component that divides large text into smaller chunks or document objects.
conversational aiA chatbot is a conversational software interface that maintains context across multiple exchanges for dialogue and interactive question answering.
memoryMemory is stored information that an agent or application can retain and use across steps or interactions.
similarity searchRetrieval is the process of finding relevant stored documents from an indexed collection in response to a query.
templateA prompt template is a reusable structure that combines user input, context, and optionally examples into a model request.
LCELLCEL is LangChain's composition language for building modular and expressive large-language-model pipelines.
LangChainLangChain is an open-source framework for building applications that combine language models with data sources, tools, and multi-step workflows.
DocumentA document is a LangChain container holding content together with metadata such as source, page number, author, or timestamp.
document loaderA data loader is a component that ingests source data into a form suitable for downstream language-model processing.
Chroma DBChroma is a vector database that stores text, embeddings, and metadata and supports similarity-based retrieval.
language modelA large language model is a neural language model trained on extensive text data to understand and generate language and perform text-based reasoning.
Runnable interfaceThe Runnable interface is a common LangChain programming interface for components that can be invoked and composed in workflows.
chat APIA chat API is an interface for exchanging conversational messages with a language model and receiving responses.
contextPrompt context is background text or information supplied alongside a request to help a language model interpret it and formulate an answer.
accuracyAccuracy is the degree to which an output, prediction, or measurement agrees with a correct or reference outcome under a specified criterion.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.