liveBook · Manning

gRPC

As explained in Designing AI Systems

gRPC is a high-performance remote procedure call framework that uses service definitions and serialized messages for communication between software components.

gRPC is a remote procedure call framework used for service-to-service communication. It lets an SDK expose network operations as function calls while generated client code handles message serialization, network calls, and responses. The calls still cross a network, even when the interface looks local. section 2.4.2

Services define methods and message schemas using Protocol Buffers. The schema is compiled into client code, giving callers an explicit interface. Binary encoding reduces the parsing and serialization overhead associated with JSON; reported comparisons are 30–50% smaller payloads and 3–5 times faster serialization. The benefit can compound when a workflow makes many internal calls per request. section 2.4.2

service SessionService {
    rpc GetOrCreateSession(GetOrCreateSessionRequest)
        returns (GetOrCreateSessionResponse);
    rpc AddMessages(AddMessagesRequest)
        returns (AddMessagesResponse);
}
 
message GetOrCreateSessionRequest {
    string user_id = 1;
    optional string session_id = 2;
}

A session service contract declares remote methods alongside the fields of a request message. section 2.4.2

An API gateway can serve external HTTPS requests while proxying internal gRPC calls. In this design, the gateway forwards binary messages without deserializing them and uses an internal registry to route calls to service instances. Clients use secure gRPC channels and identify the target service through x-target-service metadata. section 2.4.3 section 2.6

Calling the Model Service

A Python client converts message dictionaries into Protocol Buffer messages, builds a request, and calls the generated stub with routing metadata. The stub sends the request and receives the response; the client then converts that response into a Python dataclass. This keeps the workflow interface independent of Protocol Buffer objects. section 3.9.2

gRPC also supports streaming between services. A Model Service can send ChatChunk messages as token fragments arrive; the gateway converts the gRPC stream into Server-Sent Events for external clients. Following each protocol boundary helps locate streaming latency. section 3.3.1

How Designing AI Systems covers it

How deep it goes
Explains service schemas, generated clients, request conversion, gateway routing, and streaming across protocol boundaries.
Examples
Illustrates a session service contract and a model client that converts Python messages, calls a stub, and returns a dataclass.
What you can do after
Evaluate gRPC for internal service communication and understand how clients, gateways, and streaming fit together.
How it connects
Connects gRPC to Protocol Buffers, API gateways, service discovery, SDKs, and Server-Sent Events.

Also explained in

Compare how 12 books explain gRPC →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.