5 Enabling conversational memory
This chapter covers
- Maintaining conversational state
- In-memory chat history
- Retaining long-term memory
Have you seen the movie "50 First Dates"? In it, Drew Barrymore’s character, Lucy, lives with a brain injury sustained in a car accident that gives her a type of short-term memory loss. In the movie, her memory resets to the time just before her accident every day when she wakes up. She retains no memory of anything that has happened since the accident after she goes to sleep at night.
In the same movie, there’s another character called "10-Second Tom". Tom has a similar injury (from a hunting accident) in which his memory is reset every 10 seconds. Whereas Lucy’s slightly longer memory drives the plot of the movie, Tom’s condition is more comical, with him reintroducing himself by saying "Hi, I’m Tom" every 10 seconds.
Even though LLMs seem to be incredibly smart and able to answer almost any question, they have memory problems not unlike those experienced by Lucy and 10-Second Tom. In fact, Lucy’s and Tom’s memories are vastly superior the an LLM’s memory in that the LLM doesn’t retain any memory past the completion of a prompt. In short, LLMs are stateless services.