9 Explaining your ensembles

 

This chapter covers

  • Understanding glass-box versus black-box and global versus local interpretability
  • Using global black-box methods to understand pretrained ensemble behavior
  • Using local black-box methods to explain pretrained ensemble predictions
  • Training and using explainable global and local glass-box ensembles from scratch

When training and deploying models, we’re usually concerned about what the model prediction is. Equally important, however, is why the model made the prediction that it did. Understanding a model’s predictions is a critical component of building robust machine-learning pipelines. This is especially true when machine-learning models are used in high-stakes applications such as in health care or finance.

For example, in a medical diagnosis task such as diabetes diagnosis, understanding why the model made a specific diagnosis can provide users (in this case, doctors) with additional insights that can guide them toward better prescriptions, preventative care, or palliative care. This increased transparency, in turn, increases trust in the machine-learning system, allowing the users for whom the models have been developed to use them with confidence.

Understanding the reasons behind a model’s predictions is also extremely useful in model debugging, identifying failure cases, and finding ways to improve model performance. Furthermore, model debugging can also help pinpoint biases and problems with the data itself.

9.1 What is interpretability?

9.1.1 Black-box vs. glass-box models

9.1.2 Decision trees (and decision rules)

9.1.3 Generalized linear models

9.2 Case study: Data-driven marketing

9.2.1 Bank Marketing data set

9.2.2 Training ensembles

9.2.3 Feature importances in tree ensembles

9.3 Black-box methods for global explainability

9.3.1 Permutation feature importance