liveBook · Manning

Rolling summary

As explained in Designing AI Systems

A rolling summary is a continuously updated compressed representation of older conversation content that preserves its essential narrative.

A rolling summary compresses older conversation history into a compact narrative while preserving essential context. It extends effective context when prompts, tools, retrieved documents, and response tokens compete for a limited context window. section 4.7.4 section 4.8

Use the rolling summary as the middle tier of hierarchical memory:

  • Store critical, explicitly retained facts as model-managed memories in structured key-value form. section 4.7.4
  • Compress older exchanges into a few hundred tokens that retain the narrative, concerns, and prior actions. section 4.7.4 section 4.8
  • Keep recent exchanges verbatim so the current topic retains conversational nuance and flow. section 4.7.4
  • Assemble critical facts near the beginning and recent messages near the end, reflecting attention patterns that favor context-window boundaries. section 4.7.4 section 4.8

Clinical conversation

A long patient conversation can retain a rolling narrative of what brought the patient in, what has been tried, and what concerns remain, while preserving the latest exchanges word for word for follow-up questions. section 4.7.4

Figure 4.9 Hierarchical memory combines three tiers of context. Model-managed memories store critical facts as structured key-value pairs. A rolling summary compresses older conversation history into a compact narrative. Recent messages remain verbatim to preserve nuance in the current exchange.

Use the three-tier diagram to compare structured memories, compressed older context, and verbatim recent messages. section 4.7.4

How Designing AI Systems covers it

How deep it goes
Explains tier placement, compression purpose, and assembly order, but does not specify summary-generation algorithms or implementation code.
Examples
Uses a patient conversation to show preserved clinical narrative alongside verbatim recent exchanges.
What it assumes
Assumes familiarity with conversation history and token limits while contrasting summarization with simple truncation.
What you can do after
Evaluate/understand how compressed history, structured facts, and recent messages divide responsibility in long-context conversation systems.
How it connects
Connects rolling summaries to hierarchical memory, model-managed memories, truncation, retrieval-augmented memory, and context-window attention.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.