liveBook · Manning

Structured memory storage

As explained in LLMs in Production

A technique that organizes retained information so an agent can retrieve the most relevant details for subsequent steps.

Structured memory storage organizes an agent’s past into retrievable information instead of repeatedly supplying literal conversation history. The agent retrieves the details most relevant to its next step, improving performance while avoiding rapidly exhausted context windows. section 8.3

Choose it against simpler memory strategies:

  • Memory buffers pass in everything, but context limits and the “lost in the middle” problem make them unsuitable for long interactions. section 8.3
  • Memory summarization reduces history, but adds another LLM pass and can remove fine details. section 8.3
  • Structured storage requires the hardest setup, but is recommended because it retrieves the best information across scenarios. section 8.3

Retrieval pattern

Chunk articles, index or search their titles, retrieve the most relevant chunk, and chain retrievals when necessary to identify keywords or verify that the query appears in the retrieved output. section 8.3

How LLMs in Production covers it

How deep it goes
Explains the retrieval idea and trade-offs, but stops short of specifying a storage schema or implementation.
Examples
Uses article chunks, title searches, keyword retrieval, and query-containment checks as retrieval patterns.
What it assumes
Assumes familiarity with LLM agents and conversation history while treating memory strategies comparatively.
What you can do after
Evaluate memory strategies and design a retrieval-oriented approach for an agent.
How it connects
Connects memory buffers and summarization with chunking, retrieval, context limits, and external data retrieval tools.
Cover of LLMs in Production
liveBookmanning.com
Read LLMs in Production on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLMs in Production. The section links point to the parts of the book it draws on.