liveBook · Manning

Retrieval-augmented memory

As explained in Designing AI Systems

Retrieval-augmented memory retrieves relevant information from prior sessions and incorporates it into a current interaction.

Retrieval-augmented memory extends an assistant beyond the current session by treating a user's past conversations as a searchable knowledge base. When a message refers to an earlier interaction, the workflow searches prior sessions, retrieves relevant context, and adds it to the current request. section 4.7.5

Implement the pattern as a retrieval pipeline:

  • Index session messages, typically as embedding vectors, when they are added to sessions. section 4.7.5
  • Turn the current user message into a search query and search the user's conversation history through the Data Service. section 4.7.5 section 4.7.6
  • Rank and filter matches, select results within a token budget, and format them as context clearly marked as previous conversations. section 4.7.5
  • Constrain retrieval by user identity or other metadata so unrelated conversation histories are not included. section 4.7.6

Recovering a previous recommendation

A patient asks, “What did you recommend for my headaches last time?” The system searches the patient's conversation history, finds a migraine consultation from months earlier, and incorporates that session's relevant context into the current response. section 4.7.5

Figure 4.10 Retrieval-augmented memory searches past sessions for relevant context. When the patient asks about previous headache recommendations, the question triggers a search across their conversation history. Session #34 from March contains the relevant migraine consultation. That context gets pulled into the assembled context for the current session, clearly marked as coming from a previous conversation, allowing the assistant to answer accurately from months of history.

Trace the retrieval flow from a reference to a previous recommendation through session search and context assembly. section 4.7.5

Retrieval is one context-management strategy rather than a universal default. Add it when users regularly refer to earlier sessions; use simpler truncation for independent interactions and combine retrieval with summaries or model-managed memories when the application also needs long-session continuity or critical facts. section 4.7.6 section 4.8

How Designing AI Systems covers it

How deep it goes
Explains indexing, querying, ranking, filtering, formatting, and token budgeting, but defers Search Service implementation.
Examples
Uses a patient's headache recommendation, a prior migraine consultation, and a user-scoped conversation search.
What it assumes
Assumes indexed session messages and access to a Search or Data Service for user-scoped retrieval.
What you can do after
Apply it via the Data Service to answer questions that depend on relevant information from previous sessions.
How it connects
Connects RAG, semantic search, embeddings, context management, the Data Service, and hierarchical summarization.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.