A chatbot can answer a follow-up such as “Who built the temples?” without asking the user to paste the original document again. It does this by supplying earlier exchanges as context for the new question. Without that history, each turn is a separate request; remembering the conversation does not, by itself, ensure that an answer is grounded in the document. section 6.1.1 section 14.1.2
A chat history is an ordered list of messages. Each message has a role and text: a human message contains a user question, an AI message contains a response, and a system message supplies application instructions. Keep the system instructions in the prompt; track the changing human and AI messages as the conversation proceeds. section 7.4.1 section 7.4.2
To add dialogue to a retrieval-augmented generation (RAG) chatbot:
- Replace a question-and-context prompt with `ChatPromptTemplate.from_messages`. Include system instructions, a chat-history placeholder, newly retrieved context, and the current question. section 7.4.1
- Create a `ChatMessageHistory`. Add each user question and model response with `add_user_message` and `add_ai_message`, then supply its `messages` to the chain on subsequent turns. section 7.4.2 section 7.4.3
- Test with a follow-up that depends on the first answer. After asking who renamed Poseidonia, “And then what did they do?” can refer back to the Romans because the prior exchange is available. section 7.4.4
from langchain_core.prompts import ChatPromptTemplate
rag_prompt = ChatPromptTemplate.from_messages(
[
("system", """You are a helpful assistant, world-class
expert in Roman and Greek history, especially in towns
located in southern Italy. Provide interesting insights
on local history and recommend places to visit with
knowledgeable and engaging answers. Answer all questions
to the best of your ability, but only use what has been
provided in the context. If you don't know, just say you
don't know. Use three sentences maximum and keep
the answer as concise as possible."""),
("placeholder", "{chat_history_messages}"),
("assistant", "{retrieved_context}"),
("human", "{question}"),
]
)
The prompt listing shows where past messages sit relative to system instructions, retrieved context, and the new question. section 7.4.1
For a LangGraph agent, checkpoints offer another route to short-term dialogue memory. Give one conversation a `thread_id`, reuse it on each invocation, and compile the graph with a checkpointer. Unlike a message list alone, a checkpoint can retain tool outputs, intermediate values, and execution metadata alongside the conversation. `InMemorySaver` suits development; use a persistent backend when state must survive restarts or be shared across application instances. section 14.1.3 section 14.1.4
Checking whether context carries over
Ask a travel assistant about the weather in Penzance, then ask about “the same town.” With the same thread and a checkpointer, it identifies Penzance. The example’s mock weather service can return different conditions on the second call, so test reference resolution rather than matching weather values. section 14.1.5
History grows with every turn. For long conversations, keep a sliding window of recent turns or periodically summarize earlier ones to stay within token limits. Checkpointed state can also be inspected or restored to understand what context an agent had at a particular point. section 7.6 section 14.1.6
