liveBook · Manning

gRPC contract

As explained in Designing AI Systems

A gRPC contract is a formal service interface that defines the operations and interaction structure exposed between gRPC services.

A gRPC contract defines the operations a service exposes and how other services interact with it. Written in protocol buffers, it specifies RPC request and response types while leaving implementation choices—such as providers, storage, and execution locations—to the service. section 3.1.5

Group operations around service responsibilities and consumer needs. A contract can expose both ordinary responses and streaming responses: the Model Service offers Chat for response generation and ChatStream for streamed chunks. section 3.1.5

Listing 3.5 Model service gRPC definition
syntax = "proto3";
 
package ai.platform.models.v1;
 
service ModelService {
  // Core inference
  rpc Chat(ChatRequest) returns (ChatResponse);    #A
  rpc ChatStream(ChatRequest) 
➥returns (stream ChatChunk);    #A
  
  // Discovery
  rpc ListModels(ListModelsRequest) 
➥returns (ListModelsResponse);    #B
  rpc GetModelCapabilities(GetCapabilitiesRequest) 
➥returns (ModelCapabilities);    #B
  
  // Prompt management
  rpc RegisterPrompt(RegisterPromptRequest) returns 
➥(RegisterPromptResponse);
  rpc GetPrompt(GetPromptRequest) returns (Prompt);
  rpc ListPrompts(ListPromptsRequest) returns (ListPromptsResponse);
  
  // Custom model registry
  rpc RegisterModel(RegisterModelRequest) 
➥returns (RegisterModelResponse);    #C
  rpc ListRegisteredModels(ListRegisteredModelsRequest) returns 
➥(ListRegisteredModelsResponse);
  rpc GetModelStatus(GetModelStatusRequest) returns (ModelStatus);
  
  // Embeddings
  rpc Embed(EmbedRequest) returns (EmbedResponse);    #D
  rpc ListEmbeddingModels(ListEmbeddingModelsRequest)
      returns (ListEmbeddingModelsResponse);    #D
}

The Model Service definition shows RPC signatures grouped by capability, including a streaming response. section 3.1.5

Contracts can organize different platform responsibilities:

  • Model Service groups inference, model discovery, prompt management, custom model registration, and embeddings. section 3.1.5
  • Data Service exposes index management, ingestion, document management, and search, along with registration of custom parsers and chunking strategies. section 5.6
  • Tool Service supports registration, discovery, execution, and validation. Asynchronous execution returns a task ID for progress checks; validation can reject invalid arguments before execution. section 6.1.1
  • Observability Service exposes logging, metrics, tracing, scoring, cost, and health operations. It references sessions through trace session IDs rather than duplicating session lifecycle management. section 7.3
  • Experimentation Service groups target lifecycle, datasets, offline and online evaluation, A/B testing, and annotation queues. section 8.1

A contract with multiple consumers

The Workflow Service groups registry, deployment, and job operations around three consumers. A deploy CLI registers workflows and triggers deployments; a runtime server creates and updates jobs; external clients poll or cancel jobs. Their different access patterns and trust levels shape the interface. section 9.5

Figure 9.4 The three consumers of the Workflow Service contract. The deploy CLI connects to registry operations (RegisterWorkflow, UpdateWorkflow, DeleteWorkflow) and deployment operations (DeployWorkflow, RollbackWorkflow). The SDK's runtime server inside workflow containers connects to job operations (CreateJob, UpdateJobProgress, CompleteJob, SaveJobCheckpoint , FailJob). External clients connect to job operations (GetJobStatus, CancelJob) and optionally to deployment status (GetDeploymentStatus).

The consumer diagram maps each Workflow Service operation group to its callers. section 9.5

An SDK client can wrap the contract for application code. DataClient converts Python calls into gRPC requests and responses into domain objects; its search methods return lists of SearchResult objects rather than exposing protobuf messages to workflows. section 5.6

How Designing AI Systems covers it

How deep it goes
Explains service definitions and operation groups while leaving provider, storage, and execution details outside the contract.
Examples
Illustrates contracts with Model, Data, Tool, Observability, Experimentation, and Workflow Service interfaces.
What you can do after
Evaluate/understand service contracts by grouping operations around responsibilities and identifying their consumers.
How it connects
Connects protocol buffers and RPCs with SDK clients, streaming, service boundaries, and asynchronous jobs.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.