liveBook · Manning

Service-oriented architecture

As explained in Designing AI Systems

Service-oriented architecture is an architectural pattern that organizes software as independently focused services communicating through defined interfaces.

Service-oriented architecture organizes an AI platform as focused, independent services that address distinct capabilities through reusable boundaries. This replaces repeated, application-specific infrastructure with shared platform components. section 1.4

A platform can separate responsibilities into services such as:

  • Data Service for document ingestion, embeddings, and semantic search. section 1.4
  • Session Service for conversational state, preferences, and history. section 1.4
  • Workflow Service for multi-step execution, state management, retries, and failure recovery. section 1.4
  • Tool, Guardrails, Observability, Experimentation, and Model Services for integrations, safety, monitoring, evaluation, and provider abstraction. section 1.4

Context-aware support

A customer-support bot can retrieve organizational documentation through the Data Service while using the Session Service to preserve previous interactions. A refund workflow can delegate sequential and parallel operations to the Workflow Service. section 1.4

Figure 1.3 AI Platform Architecture: Our platform design directly addresses the core requirements through specialized services: the Data Service and Session Service provide context-aware intelligence; the Workflow Service enables multi-step orchestration; the Tool Service handles dynamic integrations; the Guardrails Service ensures safety and compliance; the Observability and Experimentation Services provide monitoring and experimentation; the Model Service abstracts provider differences; and the Infrastructure Layer with API Gateway provides scalable foundations. Each service operates independently while sharing common patterns, eliminating the need for teams to rebuild foundational components.

The architecture diagram maps platform requirements to specialized services and shows shared infrastructure beneath independently operated components. section 1.4

Well-defined API contracts let services scale and deploy independently. The Model Service can scale separately from the Session Service, while the Data and Workflow Services can use different storage technologies. API Gateway and Kubernetes provide discovery, configuration, resource allocation, and deployment foundations. section 1.7

How Designing AI Systems covers it

How deep it goes
Defines service responsibilities and operational boundaries, but stops short of specifying implementation code or detailed API schemas.
Examples
Uses a customer-support bot, a refund workflow, and an architecture diagram to demonstrate service collaboration.
What it assumes
Assumes multiple AI-platform requirements and applications that would otherwise rebuild shared infrastructure separately.
What you can do after
Operate/configure independently scalable services with shared contracts, discovery, resource allocation, and deployment automation.
How it connects
Connects Data Service, Session Service, Workflow Service, API Gateway, Kubernetes, service discovery, observability, and model abstraction.

Also explained in

Compare how 2 books explain service-oriented architecture →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.