This chapter covers
- Understanding glass-box versus black-box and global versus local interpretability
- Using global black-box methods to understand pretrained ensemble behavior
- Using local black-box methods to explain pretrained ensemble predictions
- Training and using explainable global and local glass-box ensembles from scratch
When training and deploying models, we’re usually concerned about what the model prediction is. Equally important, however, is why the model made the prediction that it did. Understanding a model’s predictions is a critical component of building robust machine-learning pipelines. This is especially true when machine-learning models are used in high-stakes applications such as in health care or finance.
For example, in a medical diagnosis task such as diabetes diagnosis, understanding why the model made a specific diagnosis can provide users (in this case, doctors) with additional insights that can guide them toward better prescriptions, preventative care, or palliative care. This increased transparency, in turn, increases trust in the machine-learning system, allowing the users for whom the models have been developed to use them with confidence.