gRPC is a remote procedure call framework used for service-to-service communication. It lets an SDK expose network operations as function calls while generated client code handles message serialization, network calls, and responses. The calls still cross a network, even when the interface looks local. section 2.4.2
Services define methods and message schemas using Protocol Buffers. The schema is compiled into client code, giving callers an explicit interface. Binary encoding reduces the parsing and serialization overhead associated with JSON; reported comparisons are 30–50% smaller payloads and 3–5 times faster serialization. The benefit can compound when a workflow makes many internal calls per request. section 2.4.2
service SessionService {
rpc GetOrCreateSession(GetOrCreateSessionRequest)
returns (GetOrCreateSessionResponse);
rpc AddMessages(AddMessagesRequest)
returns (AddMessagesResponse);
}
message GetOrCreateSessionRequest {
string user_id = 1;
optional string session_id = 2;
}A session service contract declares remote methods alongside the fields of a request message. section 2.4.2
An API gateway can serve external HTTPS requests while proxying internal gRPC calls. In this design, the gateway forwards binary messages without deserializing them and uses an internal registry to route calls to service instances. Clients use secure gRPC channels and identify the target service through x-target-service metadata. section 2.4.3 section 2.6
Calling the Model Service
A Python client converts message dictionaries into Protocol Buffer messages, builds a request, and calls the generated stub with routing metadata. The stub sends the request and receives the response; the client then converts that response into a Python dataclass. This keeps the workflow interface independent of Protocol Buffer objects. section 3.9.2
gRPC also supports streaming between services. A Model Service can send ChatChunk messages as token fragments arrive; the gateway converts the gRPC stream into Server-Sent Events for external clients. Following each protocol boundary helps locate streaming latency. section 3.3.1











