liveBook · Manning

Session

As explained in Designing AI Systems

A session is a structured record representing one conversation between a user and an AI application, including its messages and identifying metadata.

A session is a structured record of one conversation between a user and an AI application. It preserves the exchanges needed to interpret follow-up questions, turning otherwise stateless model calls into a continuous conversation. section 4.0 section 4.1

A session has a session ID, user ID, ordered messages, and creation and update timestamps. Each message has a role and optional content. Roles distinguish user input, assistant responses, system instructions, and tool results. section 4.1

Listing 4.2 Session dataclass containing conversation metadata and messages
@dataclass
class Session:
    session_id: str    
    user_id: str    
    messages: List[Message]    
    created_at: datetime
    updated_at: datetime    #A

The session dataclass groups identifying metadata and timestamps with the conversation's messages. section 4.1

Tool use adds structure to the message history. An assistant message can contain tool calls instead of text; each call names a function, supplies arguments, and has an ID. A subsequent tool message uses that ID to link its result to the call, including when multiple calls run in parallel. Store the calls and results alongside the other messages to preserve the conversation's sequence. section 4.1 section 4.8

Checking appointment availability

A patient asks about openings. The assistant requests a scheduling function, receives its result as a linked tool message, and then answers with the available times. One exchange therefore includes more than the visible question and answer. section 4.1

A Session Service retrieves or creates sessions, lists a user's sessions, adds messages in batches, paginates history, and deletes sessions. A storage interface separates these operations from the database. In a PostgreSQL implementation, session metadata and messages occupy separate tables; JSONB holds nested tool calls, and a message primary key provides ordering without relying on potentially colliding timestamps. section 4.8

For each request, load or create the session, retrieve history, include the relevant messages in the model request, and store the new exchange. Persistent storage lets a conversation survive infrastructure disruptions. Concurrent access also matters when a user continues a conversation across tabs or devices. section 10.3 section 4.0

Stored history can exceed the model's context window. A sliding window can send only the most recent messages while preserving older ones in storage, but the model then loses access to earlier details. Summarization and retrieval-augmented approaches are other context-management strategies. section 10.3 section 4.0

A session also groups the traces for a multi-turn conversation. Reusing its identifier connects observability data to conversation history, making it possible to inspect conversation-wide cost, turn count, and quality changes that a single request trace cannot capture. section 7.2.1

How Designing AI Systems covers it

How deep it goes
Defines session and message fields, then develops service operations, storage design, and context-window trade-offs.
Examples
Uses patient appointment scheduling and a restaurant-recommendation conversation to illustrate tool history and multi-turn continuity.
What you can do after
Builds a session-backed workflow that retrieves history, supplies context to a model, and stores exchanges.
How it connects
Connects sessions to tool calls, context windows, relational storage, and observability traces.

Also explained in

Compare how 9 books explain session →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.