liveBook · Manning

Platform

As explained in Designing AI Systems

A software and infrastructure foundation that provides shared capabilities for building and operating applications.

A platform is a shared software and infrastructure foundation for building and operating applications. For AI applications, it turns recurring production needs—monitoring, cost controls, safety, and scaling—into reusable services rather than leaving each team to rebuild them around a prototype. section 1.0 section 1.6

A layered, service-oriented architecture assigns each capability to a focused component. Services operate independently while sharing common patterns, allowing applications to combine them without owning all of their implementation details. section 1.4

Figure 1.3 AI Platform Architecture: Our platform design directly addresses the core requirements through specialized services: the Data Service and Session Service provide context-aware intelligence; the Workflow Service enables multi-step orchestration; the Tool Service handles dynamic integrations; the Guardrails Service ensures safety and compliance; the Observability and Experimentation Services provide monitoring and experimentation; the Model Service abstracts provider differences; and the Infrastructure Layer with API Gateway provides scalable foundations. Each service operates independently while sharing common patterns, eliminating the need for teams to rebuild foundational components.

The architecture maps application requirements to independent services and a shared infrastructure layer. section 1.4

Typical responsibilities include:

  • Context and orchestration: Data and Session Services supply organizational knowledge and conversational state; a Workflow Service coordinates multi-step processes. section 1.4
  • Operations and safety: observability tracks usage and performance, while guardrails and infrastructure enforce shared security patterns. section 1.1.3 section 1.6
  • Integration and governance: managed tools have names, versions, owners, and access policies; the platform handles discovery, credentials, and execution. section 6.1 section 6.3
  • Model access and experimentation: a Model Service abstracts provider differences, while experimentation supports controlled testing and performance tracking. section 1.4 section 1.1.3

An SDK can make these distributed services look like local calls. Its service clients connect through a gateway, attach routing metadata, and make gRPC calls; the gateway routes requests and the backend services perform the work. Simple defaults let applications start quickly while retaining configuration for more advanced needs. section 2.5.4 section 2.1.1

A platform also complements protocols rather than replacing them. MCP carries tool calls, while the platform manages surrounding concerns such as access policies, credentials, and invocation auditing. section 6.4.4

An assistant built on shared services

An assistant can use sessions for memory, a Data Service for organizational grounding, managed tools for actions, and guardrails for safety. Observability traces requests, while experimentation supports measured improvement. A subsequent application can reuse those services and add its own application-specific logic. section 10.13

How Designing AI Systems covers it

How deep it goes
Explains a layered service architecture and traces SDK requests through gateway routing to backend services.
Examples
Uses a patient intake assistant and a production assistant to illustrate platform-backed applications.
What you can do after
Apply it via the platform SDK to access shared services through consistent clients and gateway routing.
How it connects
Connects service-oriented architecture with observability, guardrails, workflow orchestration, tool management, and MCP.

Also explained in

Compare how 2 books explain platform →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.