liveBook · Manning

Failure mode

As explained in AI Model Evaluation

A failure mode is a recurring way in which a model or system fails under particular conditions.

A failure mode is a recurring way a model or system breaks under particular conditions. Overall quality can hide failures that cluster by task, user segment, or operating condition. section 3.4.5 section 7.6.1

Look for patterns such as:

  • A support assistant that hallucinates more on policy exceptions or gives weaker answers in some languages. section 3.4.5
  • An agent that struggles when a tool times out, returns partial data, or a task takes many steps. section 3.4.5
  • An LLM judge affected by position, verbosity, style, insufficient context, prompt injection, or overconfidence. section 9.7

From offline signal to live watchpoint

A recommendation model repeatedly puts Batman first for users without relevant watch history. Track repeat-title frequency in top recommendations by user segment; alert when it exceeds the control by more than 15% for two consecutive days. The signal can support a decision to pause, roll back, or test a patched model. section 7.6.1

Identify failure patterns in offline evaluation, translate each into an equivalent live metric, and set thresholds for alerts. Keep those watchpoints alongside experiment success metrics so known risks can be caught before an A/B test ends. section 7.6 section 7.6.1

How AI Model Evaluation covers it

How deep it goes
Explains how to identify offline failure patterns, map them to live metrics, and set alerts; it does not prescribe a universal metric.
Examples
Uses support and travel-agent tasks, LLM-judge biases, and a Batman recommendation watchpoint to illustrate failures.
What it assumes
Builds on offline evaluation and segment-level analysis, then carries known risks into live experiments.
What you can do after
Apply offline findings by defining online watchpoints, alert thresholds, and pause-or-roll-back decisions.
How it connects
Connects failure modes to offline evaluation, online monitoring, A/B testing, and LLM-as-a-judge evaluation.

Also explained in

Compare how 6 books explain failure mode →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Model Evaluation. The section links point to the parts of the book it draws on.