An API gateway provides a stable entry point to backend services while centralizing controls that would otherwise be duplicated across workflows. It authenticates requests, enforces rate limits, terminates TLS, and routes traffic to healthy containers. Services can deploy and scale independently without changing the client-facing endpoint. section 2.4.1 section 2.6
For internal calls, the gateway also acts as a gRPC proxy. SDK clients connect to it and identify the target service in request metadata; the gateway routes the call and forwards Protocol Buffer data without deserializing it. Its service registry tracks available replicas, so workflows do not need configuration changes when services scale. section 2.4.3 section 2.5.4
Figure 2.5 The Request Flow: A user's question flows from the mobile app through the API Gateway (over HTTPS), reaches the workflow container, which then makes multiple internal gRPC calls through the gateway to reach platform services. The gateway acts as a gRPC proxy, routing internal service communication while maintaining Protocol Buffer efficiency.

Follow an HTTPS request into a workflow and its internal gRPC calls through the gateway. section 2.4.3
The gateway supports production operations across independently deployed services:
- Distribute requests across healthy replicas and stop routing to failed containers. section 2.4.1
- Shift traffic gradually during deployments, with rollback if problems appear; let old containers finish in-flight requests before shutdown. section 2.4.1 section 2.4.4
- Scale gateway capacity horizontally with stateless instances behind a load balancer; each instance routes using its own registry view. section 2.4.4
- Convert model-service gRPC streams to Server-Sent Events for external clients, forwarding chunks as they arrive. section 3.3.1
Connecting a workflow SDK
A platform client resolves its gateway address in three steps: use an explicitly supplied URL, otherwise read GENAI_GATEWAY_URL, otherwise fall back to localhost:50051. This lets the same workflow code connect in development and production. section 2.5.1
Listing 2.16 GenAIPlatform constructor with gateway configuration
class GenAIPlatform:
def __init__(self, gateway_url=None): #A
if gateway_url: #B
self.gateway_url = gateway_url #B
else: #C
self.gateway_url = os.getenv("GENAI_GATEWAY_URL") #C
if not self.gateway_url: #D
self.gateway_url = "localhost:50051" #DInspect the Python constructor that implements gateway address precedence. section 2.5.1









