Exponential backoff spaces retry attempts with progressively longer delays, commonly multiplying each wait: one second, then two, then four. A maximum delay caps the growth, preventing recovery attempts from becoming unreasonably slow while reducing pressure on an impaired service. section 3.4.1
Use backoff as part of an explicit retry policy:
- Retry transient failures such as timeouts, rate limits, service unavailability, gRPC UNAVAILABLE, DEADLINE_EXCEEDED, and RESOURCE_EXHAUSTED. section 3.4.1 section 9.3.2
- Reject permanent failures such as invalid requests immediately instead of repeating the same bad operation. section 3.4.1 section 9.3.2
- Keep retry layers distinct: SDK clients recover platform-network failures, while the Model Service recovers external-provider failures. section 9.3.2
Provider retry configuration
A production-oriented configuration can allow three retries, start at one second, enable exponential growth, cap waits at 60 seconds, and retry only selected failure types. Shorter retries can support fast failover; longer retries suit batch pipelines that handle failed items separately. section 3.4.1 section 3.4.3
Listing 3.11 RetryConfig dataclass for transient failure handling
@dataclass
class RetryConfig:
max_retries: int = 3 #A
initial_delay: float = 1.0
exponential_backoff: bool = True #B
max_delay: float = 60.0
retry_on: List[FailureType] = None #C
Inspect the RetryConfig dataclass to see the policy fields for attempt limits, initial delay, exponential growth, maximum delay, and retryable failure types. section 3.4.1










