Compression condenses stored memories or knowledge into shorter representations while preserving details useful for later retrieval. It reduces redundancy by grouping related items and summarizing each group. section 8.7 section 8.9
A practical compression workflow follows these steps:
- Cluster memories or knowledge with an algorithm such as k-means. section 8.7
- Pass each group through a compression function that summarizes its contents. section 8.7
- Inspect stores for repetitive or duplicate information, especially when clusters are large or unbalanced. section 8.7
- Compare retrieval before and after compression because results depend on the use case and source verbosity. section 8.7
Compressing a knowledge store
Build a knowledge store with several documents, experiment with chunking patterns, compress the store, and observe changes in query performance. A similar experiment can compress conversational, semantic, episodic, or procedural memory stores and compare retrieval. section 8.8
Figure 8.20 The process of memory and knowledge compression
Shows the clustering-and-summarization pipeline used for memory and knowledge compression. section 8.7
