liveBook · Manning

Platform service

As explained in Designing AI Systems

A reusable infrastructure or software service provided centrally to support multiple applications or teams.

A platform service is a centrally provided capability that multiple AI applications consume instead of rebuilding independently. It turns a prototype's direct model call into a production-ready system with shared infrastructure for reliability, security, observability, cost control, and operational scale. section 1.0 section 1.2.2

A platform service consolidates recurring application requirements such as:

  • Session management for conversational state and reusable interaction infrastructure. section 1.6
  • Observability for token usage, response quality, conversation patterns, monitoring, logging, and metrics. section 1.6
  • Guardrails and infrastructure that establish consistent API-key handling, input sanitization, logging, and security defaults. section 1.6
  • Experimentation and shared configurations that let improvements in prompts or safety settings benefit multiple applications. section 1.6

Service access through an SDK

A workflow creates one GenAIPlatform object, then accesses services through clients such as platform.sessions.get_or_create(), platform.models.chat(), and platform.data.search(). The SDK connects to a gateway, service-specific clients attach routing metadata, and the gateway forwards calls to the appropriate backend. Developers write local-looking function calls while deployment, networking, and distributed-service coordination remain behind the interface. section 2.5 section 2.5.4

Figure 2.6 The SDK provides three core capabilities: the GenAIPlatform object configures gateway connections through environment variables or explicit parameters; lazy-initialized service clients (SessionClient, ModelClient, DataClient) connect to the gateway with service-specific metadata; and the @workflow decorator captures deployment configuration that the deployment tool uses to package and register workflows with the Workflow Service. This architecture enables Sarah to write simple code that hides substantial infrastructure complexity.

The architecture diagram shows how a platform object, lazy-initialized service clients, gateway connections, and workflow deployment configuration hide distributed infrastructure behind a consistent SDK. section 2.5

How Designing AI Systems covers it

How deep it goes
Explains service boundaries, shared capabilities, gateway routing, and SDK clients down to gRPC metadata, but not backend implementations.
Examples
Uses session creation, model chat, data search, patient intake workflows, unified metrics, and shared safety configurations.
What you can do after
Builds AI workflows that consume distributed services through a consistent SDK rather than implementing infrastructure separately.
How it connects
Connects session management, observability, guardrails, experimentation, gateways, SDKs, gRPC, and workflow deployment.

Also explained in

Compare how 2 books explain platform service →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.