liveBook · Manning

Interceptor

As explained in Designing AI Systems

A gRPC interceptor is a client- or server-side component that intercepts remote procedure calls to apply behavior before or after invocation.

A gRPC interceptor wraps outgoing SDK calls, applying behavior before they reach the network. For retries, it lets workflow code continue normally after a transient failure is recovered. section 9.3.2

A retry interceptor:

  • Retries UNAVAILABLE, DEADLINE_EXCEEDED, and RESOURCE_EXHAUSTED errors; other errors propagate immediately. section 9.3.2
  • Uses exponential backoff, such as delays of 100, 200, and 400 milliseconds, to avoid hammering an overloaded service. section 9.3.2
  • Operates independently of retries between a platform service and its external provider. section 9.3.2
Listing 9.13 gRPC retry interceptor for SDK service calls
import time
import grpc
 
RETRYABLE_CODES = {
    grpc.StatusCode.UNAVAILABLE,    #A
    grpc.StatusCode.DEADLINE_EXCEEDED,    #A
    grpc.StatusCode.RESOURCE_EXHAUSTED,    #A
}
 
class RetryInterceptor(
    grpc.UnaryUnaryClientInterceptor
):
    def __init__(self, max_retries=3,
                 initial_delay=0.1):
        self._max_retries = max_retries    #B
        self._initial_delay = initial_delay
 
    def intercept_unary_unary(
        self, continuation, call_details,
        request
    ):
        last_error = None
        for attempt in range(    #C
            self._max_retries + 1    #C
        ):    #C
            try:
                return continuation(    #D
                    call_details, request    #D
                )    #D
            except grpc.RpcError as e:
                last_error = e
                if (e.code() not in    #E
                        RETRYABLE_CODES):    #E
                    raise    #E
                if attempt == self._max_retries:
                    raise    #F
                delay = (self._initial_delay    #G
                    * (2 ** attempt))    #G
                time.sleep(delay)    #G
        raise last_error

The implementation defines a unary client interceptor with a retry limit and an initial delay. section 9.3.2

How Designing AI Systems covers it

How deep it goes
Implements client-call interception, retryable status filtering, and exponential delays, but does not detail server interceptors.
Examples
Demonstrates retries for UNAVAILABLE, DEADLINE_EXCEEDED, and RESOURCE_EXHAUSTED, with delays of 100, 200, and 400 milliseconds.
What you can do after
Configure automatic SDK-call retries by setting max_retries and attaching an interceptor to the gRPC channel.
How it connects
Connects gRPC channels, BaseClient, SDK service clients, exponential backoff, and service-level retries.

Also explained in

Compare how 5 books explain interceptor →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.