liveBook · Manning

Session management

As explained in Designing AI Systems

Session management is the creation, persistence, retrieval, and expiration of state associated with user or application sessions.

A chat prototype can send a model request in a few lines, but a production conversation also needs state across turns. Session management tracks interactions, persists user state, manages context windows, and supports memory across sessions. A Session Service can handle these concerns without requiring application developers to manage persistence directly. section 1.2.2 section 2.1.8

Separate the session lifecycle from conversation content. A workflow needs to find or create a session, record messages as they occur, retrieve history for a later model request, and delete the session when it is no longer needed or a user requests deletion. section 4.2

A Session Service contract exposes five operations:

  • GetOrCreateSession accepts a user ID and an optional session ID, allowing a workflow to start or resume a conversation. section 4.2
  • ListSessions returns a user’s sessions so a client can offer a conversation to resume. section 4.2
  • AddMessages records conversation content, including user and assistant messages and tool calls; GetMessages retrieves history for subsequent turns. section 4.2
  • DeleteSession provides cleanup when a conversation is no longer needed. section 4.2
Listing 4.3 Protocol Buffer service definition for the Session Service
syntax = "proto3";
 
package sessions;
 
service SessionService {
    rpc GetOrCreateSession(GetOrCreateSessionRequest) 
        returns (GetOrCreateSessionResponse);
    rpc ListSessions(ListSessionsRequest)
        returns (ListSessionsResponse);
    rpc AddMessages(AddMessagesRequest) 
        returns (AddMessagesResponse);
    rpc GetMessages(GetMessagesRequest) 
        returns (GetMessagesResponse);
    rpc DeleteSession(DeleteSessionRequest) 
        returns (DeleteSessionResponse);
}
 

The Protocol Buffer service definition groups the lifecycle and message operations into a single gRPC contract. section 4.2

The session record carries session and user IDs plus creation and update timestamps. Message records distinguish roles and can carry content, tool calls, a tool-call ID, a name, and a timestamp. History retrieval accepts optional limit, offset, and strategy fields; its response includes messages and a total count. section 4.2

Listing 4.7 The Message structure and its nested types
message Message {
    string role = 1;
    optional string content = 2;
    repeated ToolCall tool_calls = 3;
    optional string tool_call_id = 4;
    optional string name = 5;
    int64 timestamp = 6;
}
 
message ToolCall {
    string id = 1;    #A
    string type = 2;
    Function function = 3;
}
 
message Function {
    string name = 1;
    string arguments = 2;    #B
}
 

The message structure shows how conversation history can include both ordinary content and tool-call details. section 4.2

Resuming a patient conversation

A support workflow passes a user ID and, when resuming, a previous session ID to GetOrCreateSession. It records the patient’s questions, assistant responses, and tool calls, then retrieves history when preparing the next model request. If the patient requests deletion, the workflow deletes the session. section 4.2

How Designing AI Systems covers it

How deep it goes
Defines five service operations and their request and response structures, without detailing storage implementation or expiration.
Examples
Uses a patient support workflow to illustrate resuming conversations, recording messages, retrieving history, and handling deletion requests.
What you can do after
Evaluate/understand a Session Service contract for creating, listing, populating, retrieving, and deleting conversation sessions.
How it connects
Connects session management to context windows, conversation memory, tool calls, and model requests.

Also explained in

Compare how 2 books explain session management →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.