liveBook · Manning

Memory

As explained in Designing AI Systems

Model-managed memory is a capability in which an AI model selects, stores, and retrieves information from persistent memory outside its immediate context window.

Model-managed memory lets an assistant select facts worth preserving outside its immediate context window. Unlike conversation history, which records turns in sequence, memory stores structured facts that can be loaded into later requests, including requests in new sessions. The model decides what to save; the platform handles persistence. section 4.6 section 4.6.5

The context window acts like working memory, while external storage holds facts that might otherwise be lost to truncation or summarization. For example, an assistant can save a patient's penicillin allergy when it is mentioned, rather than relying on an old message to remain in context. section 4.6

A memory-enabled workflow follows three phases on each turn:

  • Load the user's saved facts and assemble them with recent conversation history and the new message. section 4.6.5
  • Generate a response with a memory tool available. The model decides whether the conversation contains a fact worth saving; the tool description guides that decision. section 4.6.5 section 10.4
  • Execute any memory tool calls and persist the conversation turn. Later requests load the saved facts without searching the original history. section 4.6.5
Listing 10.4 Giving Claw a memory tool for long-term storage
def save_user_memory(    #A
    user_id: str,    #A
    key: str,    #A
    value: str    #A
):    #A
    """Store a fact in the user's long-term memory."""
    platform.sessions.save_memory(    #B
        user_id=user_id,    #B
        key=key,    #B
        value=value    #B
    )    #B
    return {"status": "saved", "key": key}    #B
 
platform.tools.register(
    name="claw.memory.save_fact",    #C
    description=(
        "Save an important fact about the user for future reference. "
        "Use this when the user shares preferences, project details, "
        "or personal information they'd expect you to remember."
    ),
    parameters={
        "key": {
            "type": "string",
            "description": (
                "A short identifier for this fact, e.g. "
                "'preferred_meeting_time', 'current_project', "
                "'dietary_restrictions'"
            )
        },
        "value": {
            "type": "string",
            "description": (
                "The fact itself, e.g. 'morning' or "
                "'Project Atlas'"
            )
        }
    },
    handler=save_user_memory    
)

A registered tool delegates storage to the Session Service and tells the model when saving a fact is appropriate. section 10.4

A memory entry has a key, a flexible value, a required user ID, an optional session ID, and timestamps. The session ID can identify the originating conversation or support session-scoped retrieval. Applications decide whether memories expire or survive session deletion. Unlike an append-only message log, a memory can be updated when a fact changes. section 4.6.2 section 4.6.1

Concurrent sessions can update the same user-level key. Last-write-wins is a simple default; where overwrites could lose important information, an updated-at check can detect conflicts for retry or resolution, or the application can append rather than overwrite. section 4.6.2

Memory complements rather than replaces context management. A hierarchical context can combine structured facts, a rolling summary of older conversation, and verbatim recent messages. Choose the layers according to user behavior: independent requests may need only truncation, while longer conversations or references to previous sessions may justify summarization or retrieval. section 4.7.4 section 4.7.6

Figure 4.9 Hierarchical memory combines three tiers of context. Model-managed memories store critical facts as structured key-value pairs. A rolling summary compresses older conversation history into a compact narrative. Recent messages remain verbatim to preserve nuance in the current exchange.

The three-tier layout separates durable facts from compressed history and recent conversational detail. section 4.7.4

How Designing AI Systems covers it

How deep it goes
Develops the memory data model, tool workflow, scoping, concurrency choices, and integration with context management.
Examples
Uses a healthcare assistant saving allergies and medications, and a personal assistant storing travel preferences and project details.
What you can do after
Apply model-managed memory via the platform's Session Service and registered memory tools in an assistant workflow.
How it connects
Connects model-managed memory to tool calling, conversation history, summarization, hierarchical memory, and retrieval-augmented memory.

Also explained in

Compare how 26 books explain memory →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.