A fallback is an alternative response used when a preferred model, service, or processing path is unavailable, too slow, or fails. It protects user experience during outages and performance degradation, but it should remain a safety mechanism rather than the normal product experience. section 4.1 section 4.4.2
Evaluate fallback behavior as part of the complete serving system:
- Define when the system switches paths, such as after a response-time threshold, model failure, timeout, or downstream service failure. section 4.0 section 4.4.2
- Measure how often fallback responses occur and whether they degrade functionality or user experience. section 4.4.2
- Log experiment assignment, model version, output, fallback path, latency, errors, and downstream outcomes so fallback effects are not mistaken for model effects. section 6.4.3
- Treat frequent fallback activation as a diagnostic signal: inspect retrieval, inference, integrations, capacity, and tail latency rather than accepting it as the target state. section 4.1 section 4.4.2
Recommendation serving under load
A recommendation service may return a default or static ranking when real-time predictions cannot be fetched or processed within an acceptable time. Monitoring fallback frequency alongside latency, errors, and user outcomes reveals whether the service is preserving availability or routinely hiding a performance problem. section 4.1 section 4.4.2























