liveBook · Manning

Conversational ai

As explained in Designing AI Systems

A personal AI assistant is an AI application that helps an individual by understanding requests, providing information, and performing actions on the user's behalf.

Conversational AI lets an assistant understand requests and respond in natural language. A personal assistant can also remember preferences, search documents, and take actions on a user’s behalf. Moving from a convincing prototype to a reliable assistant requires more than a model call: concurrency, safety, current knowledge, cost controls, and visibility all matter. section 1.1.1 section 10.0

A platform separates application-specific behavior from the services that support it. The workflow orchestrates requests; session storage supplies conversation history and persistent memory; retrieval provides organizational knowledge; and the model generates responses. Managed tools enable actions, while guardrails, observability, and experimentation support safe operation and improvement. section 10.0 section 10.14

Figure 1.4 The conversational request flow shows how Maria's simple question flows through multiple platform services before returning a response. The Workflow Service orchestrates the entire interaction, coordinating calls to the Session Service to retrieve conversation history, the Guardrails Service to validate both input and output for safety, the Data Service to search organizational knowledge, and the Model Service to generate contextually appropriate responses. The Observability Service tracks costs and performance throughout this coordinated sequence, demonstrating how platform services transform simple interactions into production-ready AI applications.

Follow a customer question through orchestration, conversation history, safety checks, knowledge retrieval, generation, and observability. section 1.5

Context engineering assembles information from multiple sources into a coherent context window, with priorities, budget allocations, and degradation strategies. For assistants that act, an agent loop repeats reasoning, action, and observation. Managed tool execution, credential isolation, and rate limiting help constrain that loop; blocked actions can be returned to the model as tool results rather than causing abrupt failure. section 10.14

Grounding a healthcare assistant

An assistant answering insurance questions needs current provider information, not generic responses from model training data. It can search an organizational index, include the retrieved text in the model request, and return document identifiers with the answer. section 2.1.3

Listing 2.3 Integrating organizational knowledge through Data Service
relevant_info = platform.data.search(    #A
    query=question,    #A
    index_name="patient_procedures"    #A
)
relevant_text = "\n".join(r.text for r in relevant_info)    #A
 
response = platform.models.chat(
    model="gpt-4o",
    system_prompt_name="patient_intake_assistant",
    messages=[    #B
        {"role": "system",    #B
         "content": f"Relevant information:\n{relevant_text}"},    #B
        {"role": "user", "content": question}],    #B
)
return {
    "response": response.content,
    "sources": [r.document_id for r in relevant_info]    #C
}

The listing shows the search-to-response interface and returns source document identifiers alongside the answer. section 2.1.3

A production assistant also needs a way to measure what happens. Usage tracking and cost attribution expose spending, while traces, quality scoring, versioning, and experiments support systematic improvement. These controls address failures that may not appear in a small demonstration. section 1.1.1 section 10.14

How Designing AI Systems covers it

How deep it goes
Develops an assistant incrementally through service coordination, context engineering, an agent loop, safety checks, deployment, and observability.
Examples
Traces a retail return-policy question and illustrates a healthcare assistant searching organizational documents and returning source identifiers.
What you can do after
Builds a personal assistant by coordinating platform services for memory, retrieval, actions, safety, and measurement.
How it connects
Connects conversational AI to retrieval-augmented generation, context engineering, agent loops, guardrails, observability, and experimentation.

Also explained in

Compare how 26 books explain conversational ai →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.