A stateless system processes each interaction from scratch without retaining information from earlier interactions. This suits one-off questions and simple searches, but multistep applications need memory to preserve continuity and build on earlier results. section 9.2.3
Memory becomes important when an application must carry information forward:
- Short-term memory stores prompt context, activity traces, summaries, or selected relevant interactions; larger contexts increase latency, token usage, and possible inaccuracies. section 9.2.3
- Long-term memory stores interactions, results, and retrieved information in an external database for later tasks and personalization. section 9.2.3
From a question to a roadmap
A one-off search can reset after answering. An agent updating a product roadmap, however, must carry trend research into synthesis and may retrieve prior-quarter interactions from long-term memory. section 9.2.3
Figure 9.14 Agents need short-term memory for the current session and long-term memory for learning and improvement over time.
Compare short-term session memory with long-term memory used for learning and improvement over time. section 9.2.3
