4 The Session Service: Teaching your AI to remember
This chapter covers
- Designing the Session Service to store and retrieve conversation history
- Defining the gRPC contract and connecting the service through the SDK
- Abstracting session storage and implementing a PostgreSQL backend
- Extending the Session Service with model-managed memory
- Managing context windows with strategies like summarization, hierarchical memory, and retrieval-augmented approaches
In this chapter we will build the Session Service, one half of what chapter 1 called "context-aware intelligence" (the Data Service, which handles organizational knowledge, is the other half). The Session Service provides conversation memory: the ability to remember what's been said so that follow-up questions make sense and the assistant can reference earlier parts of the conversation. This capability transforms a stateless AI system into something genuinely useful. When a patient asks, "What documents do I need?" and then follows up with "What about for my child?", the assistant understands that "what" refers to documents because it remembers the previous exchange.
Sessions are structured and relational. A user has conversations, conversations have messages, messages have roles and content. We look up sessions by ID and append new messages in sequence. Traditional databases handle this naturally, which means the patterns in this chapter will feel familiar if you've worked with any backend system that stores user data.