A chatbot without memory treats every question as a fresh request. After answering who renamed Poseidonia to Paestum, it may fail to understand “What did they do next?” because it cannot connect “they” to the Romans. Memory retains information from earlier interactions so an assistant can handle follow-ups and sustain a conversation. section 7.3.4 section 7.4.4
Choose the scope of memory before implementing it:
- Short-term memory keeps context within one session, such as the town named in an earlier weather question. section 14.1.1
- Long-term user memory persists across sessions for one person, such as preferred destinations. Long-term application memory persists information useful across users, such as travel updates. section 14.1.1
For a LangChain RAG chatbot, retrieve the chat history, insert it into the prompt, and update it after each exchange. LCEL connects that history with the retriever, prompt, and model. The resulting history lets the chatbot interpret a vague follow-up using an earlier answer rather than treating it as an isolated question. section 7.4.4
For a LangGraph agent, a checkpointer provides short-term conversational memory by saving agent state. Checkpoints are taken after node executions and can contain conversation history, tool outputs, and intermediate reasoning. They also support resuming a workflow after an interruption and branching from an earlier checkpoint. section 14.1 section 14.4
A follow-up that needs context
A travel assistant answers “What’s the weather like in Penzance?” When asked “What’s the weather in the same town now?”, it resolves “same town” to Penzance using conversation history. The example uses a mock weather service that returns random conditions, so different weather readings do not demonstrate a memory failure. section 14.1.5
Table 14.1 Type of memory in AI agents
|
Memory type
|
Scope
|
Persistence
|
Example (travel assistant)
|
Challenges
|
|---|---|---|---|---|
|
Short-term
|
Single user session
|
Until session ends
|
Remembering “same town” in a weather follow-up within one conversation
|
Limited continuity; lost when session closes
|
|
Long-term (user)
|
Across multiple sessions for one user
|
Weeks, months, or years
|
Remembering a user’s preferred Cornwall destinations or accommodation types
|
Privacy compliance and managing the persisted data
|
|
Long-term (application)
|
Across all users and sessions
|
Ongoing
|
Storing general travel updates (e.g., Cornwall event calendar, seasonal attraction schedules)
|
Keeping data fresh; avoiding outdated or incorrect information
|
Compare the persistence, travel-assistant use case, and operational challenge of each memory scope. Short-term context ends with the session; persistent memory raises data-management, freshness, and privacy concerns. section 14.3.1
Start with short-term memory when the immediate problem is conversational continuity. Persisting preferences or shared knowledge is a separate design task: long-term memory may need per-user stores, summarization or pruning, and privacy controls. It can improve personalization, but adds scaling and regulatory challenges. section 14.3.1
