liveBook · Manning

Interpretability

As explained in Evaluating AI Systems

The extent to which a system’s behavior, outputs, or evaluation measures can be made understandable to people.

Interpretability makes evaluation metrics understandable to the people who use them. They should be able to tell what a metric measures, why it matters, and how to interpret its results. section 3.3.7

A rigorous number can still be difficult to act on if its meaning is unclear. This matters especially in high-stakes settings, where unclear metrics can weaken accountability, invite regulatory pushback, and erode public trust. section 3.3.7

Choose and communicate metrics with their users and purpose in mind:

  • For operational or stakeholder-facing evaluations, prioritize intuitive measures tied to observable system behavior. section 3.3.7
  • For research or diagnosis, use specialized metrics where needed, and explain complex measures with examples or visualizations. section 3.3.7
  • Assess interpretability alongside other metric qualities; no single metric captures every aspect of system behavior or failure. section 3.7

Making recall meaningful

A recall of 0.85 for a medical diagnosis system can be translated as correctly identifying 85% of cancer cases while keeping false alarms to 10%, making the result more useful to clinicians. section 3.3.7

How Evaluating AI Systems covers it

How deep it goes
Defines interpretability as a metric-quality criterion and offers selection guidance, but no formal measurement procedure.
Examples
Translates cancer-detection recall into cases identified and false alarms, and contrasts stakeholder-facing with research metrics.
What you can do after
Evaluate whether users can understand what a metric measures, why it matters, and how to interpret its results.
How it connects
Connects interpretability with evaluation metrics, accountability, regulatory compliance, and human judgment.

Also explained in

Compare how 24 books explain interpretability →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Evaluating AI Systems. The section links point to the parts of the book it draws on.