liveBook · Manning

Memory

As explained in Build a Multi-Agent System (from Scratch)

Memory is a collection of records from an agent's experiences that can influence its future behavior.

Memory gives an LLM agent access to records of its own experiences so they can influence future tasks. Because LLMs are stateless, recalled information must be supplied in the agent’s context. Useful records can include task inputs, results, tool calls, trajectories, and lessons from failures. section 7.1.2 section 7.1.1

Reusing a previous result

An agent that has computed the hailstone sequence for 5 has already encountered the sequence starting at 16. Recalling the earlier result can avoid repeating nearly all the tool calls needed for the second task. section 7.1.1

Two kinds of memory serve different points in the task lifecycle:

  • Working memory tracks progress within one task. The TaskHandler’s rollout is injected into the system message and updated after each step; it is transient and limited by the context window. section 7.1.2
  • Episodic memory records completed task executions for recall on future tasks. An Episode is the unit of memory; Memory records and recalls episodes through a BaseMemoryStore backend. section 7.0 section 7.7

Memory is defined by where records come from, not by a particular storage method. User preferences, skills, model weights, and retrieval over Wikipedia are outside this experience-based definition. section 7.1.2

Figure 7.2 Memory records and recalls Episode objects through its BaseMemoryStore backend. LLM agents use these operations within its task execution lifecycle.

The diagram connects Episode records, the Memory interface, its storage backend, and the agent lifecycle. section 7.1.2

An LLMAgent can hold multiple Memory instances. At task start, TaskHandler.load_memories() recalls from each one and formats the results for injection into the system message used by run_step(). At task end, record_memory() builds an Episode from the task, rollout, and result or error, then writes it to the configured memories. A failed recall is logged and does not prevent attempts to use other memories. section 7.5.1 section 7.5.2

Storage and recall choices shape the behavior of episodic memory:

  • recency_memory uses JSONMemoryStore, a local JSONL-backed store designed for recent-record recall. section 7.4.2
  • similarity_memory uses QdrantMemoryStore for similarity-based recall; the recall method and storage backend must be compatible. section 7.4.2 section 7.7
  • reflective_memory uses an LLM to derive a lesson from an episode’s result or error before storing it. Its metadata_fns hook enables this enrichment, and its reflection template can be customized. section 7.4.2

In a multi-agent system, agents can keep individual memories or share a store. Shared episodes should identify the recording agent, and shared writes require care: stale file versions can overwrite records. A concurrency-safe store or a coordinator responsible for memory management can address this risk. section 7.6

How Build a Multi-Agent System (from Scratch) covers it

How deep it goes
Implements episodic memory and lifecycle integration, while leaving recall during a task sub-step out of scope.
Examples
Uses hailstone sequences to motivate reuse and recency, similarity, and reflection recipes to demonstrate memory designs.
What it assumes
Builds on an existing agent framework with tools and skills while introducing memory fundamentals.
What you can do after
Builds episodic memory from scratch and integrates recall and recording into an agent’s task lifecycle.
How it connects
Connects memory to context windows, tool calls, retrieval-augmented generation, and multi-agent systems.

Also explained in

Compare how 26 books explain memory →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build a Multi-Agent System (from Scratch). The section links point to the parts of the book it draws on.