Semantic memory stores and retrieves facts, concepts, and contextual information by meaning rather than only literal terms or explicit graph relationships. It is a form of long-term memory for an agent or user, complementing short-term conversational memory. section 6.4.1
Its basic pattern resembles retrieval-augmented generation: encode memories, index them, search for related meaning, and retrieve results to augment the agent’s context. Memories may differ from complete documents, so splitting or chunking is not always necessary. section 6.4.1
Choose the storage and retrieval pattern according to what must be remembered:
- Use semantic or vector storage for broad meaning-based retrieval of facts, concepts, and contextual information. section 6.4.1
- Use keyword or relational databases when precise facts and statistics are the main target. section 6.4.1
- Use graph storage when relationships among people, entities, or observations matter. section 6.4.1
- Augment episodic or procedural memories with semantic meaning by generating questions that could later retrieve them. section 6.4.4
Hybrid semantic and graph memory
A hybrid agent can search a ChromaDB semantic store for documents matching a user’s meaning, extract entities and observations, search graph memory for those items, update both stores, and respond from the combined results. This improves relevance, but local examples may be ephemeral; production systems require persistence, access control, observability, stale-fact eviction, and retrieval evaluation. section 6.4.3
Figure 6.9 Basic semantic memory retrieval and augmentation workflow. At the top of the figure, memories are stored as semantic encodings in a vector database. At the bottom of the figure, we can see the basic RAG process for retrieving and augmenting agent context.
Trace the core workflow from semantic encodings in a vector database through retrieval and agent-context augmentation. section 6.4.1
Figure 6.11 Semantic memory augmentation works by feeding previous experiences into an LLM. The LLM then converts each experience into an augmented memory consisting of questions a user may ask to activate that memory in a future retrieval step.
Inspect how an LLM turns previous experiences into questions that can activate those memories during future retrieval. section 6.4.4
