liveBook · Manning

Chapter 10: Building an AI assistant: putting the platform to work

What you will learn in chapter 10 of Designing AI Systems

This introduction frames the chapter as an incremental build of Claw, a complete AI assistant that composes every platform service while keeping application code focused on assistant behavior.

After this chapter you will be able to

What it builds on

Section by section

Introduction

This introduction frames the chapter as an incremental build of Claw, a complete AI assistant that composes every platform service while keeping application code focused on assistant behavior.

Key terms: Model Service, Session Service, Data Service, Tool Service, Guardrails Service, Observability Service, Claw, agent loop, platform service, persistent memory

10.1 The blueprint

This section previews Claw’s complete architecture, showing a small workflow orchestrating platform services and the incremental path to assembling it.

Key terms: API Gateway, Claw workflow container, workflow function, context window, Model Service, streaming, session, long-term memory, knowledge base, tool calls

10.2 The simplest assistant: a model call in a workflow

This section builds Claw’s minimal workflow-wrapped model call, shows what the platform handles for it, and identifies its lack of conversational memory.

Key terms: workflow function, Model Service, model call, Claw, SDK, provider resolution, retries, fallbacks, request formatting, system prompt

10.3 Teaching Claw to remember conversations

This section gives Claw within-session conversational continuity by persisting exchanges through the Session Service while managing finite model context windows.

Key terms: Session Service, persistent session, conversation history, context window, exchange, session_id, get_or_create, PostgreSQL, sliding window, most recent N messages

10.4 Long-term memory: remembering across sessions

This section gives Claw persistent, model-managed user memory across sessions by selectively storing facts and injecting them into each request's context.

Key terms: Session Service, conversation history, long-term memories, model-managed memory, current session, memory tool, Tool Service, save_fact, tool description, prompt engineering

10.5 Grounding Claw in reality: knowledge retrieval and agentic RAG

This section grounds Claw in organizational knowledge by progressing from fixed retrieval-augmented generation to model-directed, agentic RAG.

Key terms: retrieval-augmented generation (RAG), Data Service, platform.data.search, context window, knowledge base, score threshold, semantic match, token budget, source attribution, document_id

10.6 From conversation to action: tools and the agent loop

This section equips Claw to act through managed tools using a bounded agent loop that reasons, executes, observes results, and continues until it can answer.

Key terms: Tool Service, platform.tools.execute, handler-backed tools, endpoint tools, MCP tools, tool registration, credential_ref, Credential Store, is_read_only, is_idempotent

10.7 Safety: guardrails at every step

This section makes Claw safe through layered input, tool-argument, and output guardrails integrated into its agent loop.

Key terms: Guardrails Service, defense-in-depth, input validation, tool argument validation, output filtering, topic policy, block action, calendar.create_event, tickets.create, session-scoped limit

10.8 The prompt that makes it all work: context engineering

This section explains how Claw assembles competing static and dynamic information sources into a token-budgeted, application-specific model context.

Key terms: context engineering, context window, token budget, user memory, conversation history, retrieved documents, tool descriptions, system prompt, Prompt Registry, docs-versus-history trade-off

10.9 The complete agent loop

This section assembles Claw’s previously built capabilities into a complete, platform-backed agent loop and distinguishes the engineering layers around the model.

Key terms: agent loop, input guardrails, context assembly, tool calls, iteration limit, output filtering, agent, pipeline, branching decisions, agentic behavior

10.9.1 Streaming the response

This subsection adapts the Claw agent loop to stream safe, user-facing progress and response tokens while preserving its existing tool execution and platform responsibilities.

Key terms: chat_stream, chat, agent loop, tool_call_delta, finish reason, token events, status events, ephemeral indicators, filter_output, filter_output_stream

10.10 Seeing what Claw does: observability

This section shows how Claw’s automatic platform telemetry and quality scoring make production behavior measurable and actionable.

Key terms: gRPC middleware, Observability Service, trace tree, span, Model Service, Data Service, latency, cost breakdown, caching, external dependency

10.11 Making Claw better: experimentation

This section shows how Claw uses a statistically controlled A/B test to decide whether a prompt change improves production behavior.

Key terms: Observability Service, Experimentation Service, quality scores, A/B test, tools-first prompt, current prompt, sticky assignment, sample-size guarantees, control group, variant

10.12 Deploying Claw

This section deploys Claw through the Workflow Service and shows how stateless scaling, asynchronous jobs, and parallel child workflows support production research tasks.

Key terms: @workflow decorator, deployment contract, platform deploy, Workflow Service, container image, Kubernetes deployment, /claw/chat, autoscaler, replicas, stateless

10.13 What the platform gave us

This section concludes that a shared AI platform turns a small model-call workflow into a production-grade assistant while making future AI applications cheaper to build through service reuse.

Key terms: Claw, workflow, model call, production-grade assistant, platform service, application-specific logic, sessions, organisational documents, managed credentials, rate limits

10.14 Summary

This summary shows how Claw combines the platform’s services into a production-ready AI assistant, with context engineering, safe agent actions, and built-in improvement capabilities.

Concepts explained in this chapter

← Chapter 9: The Workflow Service: orchestrating and deploying AI applications

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.