A fallback is an alternative execution or service path used when a preferred path fails, becomes unavailable, or produces an unsatisfactory result. In GenAI systems, it helps preserve availability and response quality while handling retrieval uncertainty, downtime, API limits, and changing query needs. section 7.7.3 section 10.2.2
Design fallback behavior around explicit signals and operational controls:
- Retry retrieval with different chunk sizes or chunking strategies when the initial response falls below a confidence threshold. section 7.7.3
- Use user feedback as a signal for another retrieval attempt, while recognizing that repeated processing can increase cost and complexity. section 7.7.3 section 10.2.2
- Pair automatic recovery with error logging, anomaly alerts, downtime planning, and distributed tracing. section 10.2.2
Retrieval fallback
A retrieval pipeline first searches using its normal chunking configuration. If the generated answer does not meet its confidence threshold, it re-queries with a different chunk size or chunking strategy; unsatisfactory user feedback can trigger another attempt. section 7.7.3
Figure 10.3 Generative AI architecture principles
Use the architecture-principles diagram to place fallback beside error handling, monitoring, security, cost control, and user experience concerns. section 10.2.2
