A runtime fallback redirects a failed request to an alternative provider while the application is running. It differs from a configuration default: a default selects the provider before execution, whereas fallback changes providers after failure. section 6.4.3
Choose runtime fallback deliberately because it changes both availability and the meaning of operational measurements.
- Use fallback to improve availability when a primary provider returns an operational failure, such as an overloaded-model 503. section 6.4.5
- Avoid fallback when benchmark purity matters: silently switching providers can contaminate precision metrics with unmeasured secondary-provider results. section 6.4.3
- Keep provider selection explicit through a client abstraction and provider configuration when isolated routing is more important than seamless recovery. section 6.4.3
An overloaded provider during testing
A Gemini request returns 503 because the model is overloaded under free-tier rate limits. A production-oriented system could route that request to OpenAI, but an evaluation-oriented system lets the error reach the user so benchmark data remains isolated. section 6.4.5
Figure 6.7 Text-to-SQL in Cloud Run Logs.

Inspect the cloud log evidence for the selected Gemini provider and the failed request that exposes the fallback decision. section 6.4.5
