chapter six

6 Adding memory to your agent

 

This chapter covers

  • Understanding the role of memory in LLM agents
  • Managing context growth with sliding window, compaction, and summarization
  • Implementing sessions for multiturn conversations
  • Building asynchronous human-in-the-loop workflows
  • Creating long-term memory for cross-session knowledge retention

Memory is what separates a stateless tool from an intelligent assistant. Without memory, an agent can’t recall previous events within the same task, continue conversations from earlier sessions, or learn from experiences. Each interaction starts from scratch, forcing users to repeat context and preventing the agent from improving over time.

This chapter (figure 6.1) addresses memory in three use patterns:

  • We implement context optimization strategies to prevent context explosion, in which the context takes up a large share of the model’s context window or grows past it during complex problem solving.
  • We build Session and SessionManager to maintain conversation continuity across multiple interactions, extending this architecture to support asynchronous human-in-the-loop (HITL) workflows.
  • We create a long-term memory system that extracts, stores, and retrieves knowledge across session boundaries using vector search.
Figure 6.1 Book structure overview: chapter 6

6.1 The anatomy of agent memory

6.1.1 Limitations of the current memory architecture

6.1.2 Three challenges of the current memory architecture

6.1.3 Context engineering and memory

6.2 Managing context during execution

6.2.1 Separating storage from presentation

6.2.2 Sliding-window strategy

6.2.3 Token counting

6.2.4 Compaction strategy

6.2.5 Summarization strategy

6.2.6 Hierarchical context management

6.3 Continuous execution: Session and state management

6.3.1 The Session class

6.3.2 Managing sessions with SessionManager