A taxi fare that suddenly rises may be accurate, but the rider still needs to know why. Interpretability concerns understanding a model’s internal mechanics—how and why it produces predictions. Explainability is related but distinct: it expresses model behavior in terms people can understand, even when those mechanics are opaque. section 11.2
Treat interpretability as a design criterion throughout the ML lifecycle:
- Compare baselines on accuracy, development effort, computation time, and interpretability. Extra complexity needs a worthwhile gain. section 8.5
- Select features for predictive value and clarity. Complex features can capture more nuance but make predictions harder to explain and pipelines harder to maintain. section 11.1.1
- Analyze feature importance to identify major contributors to predictions. Keep the results with training artifacts; they may inform model verification before deployment. section 11.2
- Plan for explanation needs before release, including regulatory requirements or an admin interface; adding those capabilities later may be expensive. section 13.3.2
A retail forecasting decision
A feature that significantly improves forecasts is not automatically worth adding. If it takes two days to compute or depends on fragile data sources, a simpler, easier-to-describe feature may be the better system choice. section 11.5.1
Figure 11.2 An example of a taxi aggregator app’s UI that clarifies why its dynamic pricing algorithm chooses this particular price in this area and during this time of the day
The taxi pricing interface illustrates the difference between producing a price and giving the rider an understandable reason for it. section 11.2
There is no universal accuracy–interpretability setting. Highly complex models can be difficult to understand despite strong performance; choose the balance for the problem, the people affected, and the required accuracy. section 11.2.2
