A fallback strategy keeps an application available by defining an alternative provider, model, or execution path when the preferred path fails or cannot satisfy a request. In LLM systems, fallback complements retry logic, monitoring, and graceful degradation. section 12.2.3 section 13.4.6
Production fallback planning includes:
- Keep OpenAI or Claude available as backup options when Gemini availability or capacity becomes a concern. section 12.2.3
- Combine fallback with exponential backoff for temporary throttling, monitoring for token usage, and broader graceful-degradation strategies. section 12.2.3 section 13.4.6
Provider fallback for a hosted LLM
A high-availability application uses Gemini as its preferred provider, tracks embedding and generation token usage, retries temporary throttling with exponential backoff, and maintains OpenAI or Claude as backup options. section 12.2.3
