liveBook · Manning

Fallback

As explained in AI Model Evaluation

An alternative system behavior or response used when a preferred processing path or primary component is unavailable or fails.

A fallback is an alternative response used when a preferred model, service, or processing path is unavailable, too slow, or fails. It protects user experience during outages and performance degradation, but it should remain a safety mechanism rather than the normal product experience. section 4.1 section 4.4.2

Evaluate fallback behavior as part of the complete serving system:

  • Define when the system switches paths, such as after a response-time threshold, model failure, timeout, or downstream service failure. section 4.0 section 4.4.2
  • Measure how often fallback responses occur and whether they degrade functionality or user experience. section 4.4.2
  • Log experiment assignment, model version, output, fallback path, latency, errors, and downstream outcomes so fallback effects are not mistaken for model effects. section 6.4.3
  • Treat frequent fallback activation as a diagnostic signal: inspect retrieval, inference, integrations, capacity, and tail latency rather than accepting it as the target state. section 4.1 section 4.4.2

Recommendation serving under load

A recommendation service may return a default or static ranking when real-time predictions cannot be fetched or processed within an acceptable time. Monitoring fallback frequency alongside latency, errors, and user outcomes reveals whether the service is preserving availability or routinely hiding a performance problem. section 4.1 section 4.4.2

How AI Model Evaluation covers it

How deep it goes
Defines fallback triggers and observability needs, then stops short of prescribing a specific fallback implementation.
Examples
Uses default or static recommendation rankings when real-time predictions cannot be fetched or processed acceptably.
What you can do after
Diagnose fallback frequency by correlating paths with latency, errors, bottlenecks, and downstream user behavior.
How it connects
Connects fallback behavior to latency, load time, model serving, A/B testing, logging, and downstream user outcomes.

Also explained in

Compare how 24 books explain fallback →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Model Evaluation. The section links point to the parts of the book it draws on.