A session is a structured record of one conversation between a user and an AI application. It preserves the exchanges needed to interpret follow-up questions, turning otherwise stateless model calls into a continuous conversation. section 4.0 section 4.1
A session has a session ID, user ID, ordered messages, and creation and update timestamps. Each message has a role and optional content. Roles distinguish user input, assistant responses, system instructions, and tool results. section 4.1
Listing 4.2 Session dataclass containing conversation metadata and messages
@dataclass
class Session:
session_id: str
user_id: str
messages: List[Message]
created_at: datetime
updated_at: datetime #AThe session dataclass groups identifying metadata and timestamps with the conversation's messages. section 4.1
Tool use adds structure to the message history. An assistant message can contain tool calls instead of text; each call names a function, supplies arguments, and has an ID. A subsequent tool message uses that ID to link its result to the call, including when multiple calls run in parallel. Store the calls and results alongside the other messages to preserve the conversation's sequence. section 4.1 section 4.8
Checking appointment availability
A patient asks about openings. The assistant requests a scheduling function, receives its result as a linked tool message, and then answers with the available times. One exchange therefore includes more than the visible question and answer. section 4.1
A Session Service retrieves or creates sessions, lists a user's sessions, adds messages in batches, paginates history, and deletes sessions. A storage interface separates these operations from the database. In a PostgreSQL implementation, session metadata and messages occupy separate tables; JSONB holds nested tool calls, and a message primary key provides ordering without relying on potentially colliding timestamps. section 4.8
For each request, load or create the session, retrieve history, include the relevant messages in the model request, and store the new exchange. Persistent storage lets a conversation survive infrastructure disruptions. Concurrent access also matters when a user continues a conversation across tabs or devices. section 10.3 section 4.0
Stored history can exceed the model's context window. A sliding window can send only the most recent messages while preserving older ones in storage, but the model then loses access to earlier details. Summarization and retrieval-augmented approaches are other context-management strategies. section 10.3 section 4.0
A session also groups the traces for a multi-turn conversation. Reusing its identifier connects observability data to conversation history, making it possible to inspect conversation-wide cost, turn count, and quality changes that a single request trace cannot capture. section 7.2.1








