Explainable Boosted Machine

As explained in Ensemble Methods for Machine Learning

An explainable boosting machine is an additive ensemble model that combines boosted shallow-tree components for individual features and selected feature pairs.

An explainable boosted machine (EBM) is a glass-box ensemble built to make predictions and explanations part of the same model. Rather than train a black box and explain it afterward, an EBM represents a prediction as a sum of nonlinear effects from individual features plus a small set of important feature pairs. Each learned component can be inspected directly for global behavior or for its contribution to one prediction. section 9.5 section 9.5.1

The additive structure starts with a generalized additive model (GAM): each feature gets its own nonlinear function. An EBM extends that structure with selected two-feature functions, rather than every possible pair. Limiting interactions avoids model bloat while retaining interactions the model finds effectual; because every component depends on only one or two variables, its relationship to the target is directly visualizable. section 9.5.1

Training proceeds in two stages:

• Cycle through features repeatedly, fitting one shallow tree component per feature on each pass. Very low learning rates and many iterations gradually learn nonlinear single-feature effects. section 9.5.1 section 9.5.1
• Repeat a similar cyclic procedure for selected pairwise interactions, adding only a small number of interaction components to the additive model. section 9.5 section 9.5.1
• Sum the learned component scores to obtain a prediction; inspect those same scores as positive or negative feature and interaction contributions for an individual example. section 9.5 section 9.5.2

Figure 9.21 An EBM is a generalized additive model consisting of nonlinear components that depend on only one feature as well as nonlinear components that depend on pairs of features. This example shows an EBM for diabetes diagnosis dependent on three variables: age, glc, and bmi. Though there are three pairs of variables (age-glc, glc-bmi, age-bmi), this EBM includes only one of them that it has deemed significant. The explainable boosting model is also an ensemble.

This diabetes-diagnosis layout makes the selective-interaction idea concrete: age, blood glucose, and BMI each have individual components, while only one of three possible feature pairs is retained. section 9.5.1

Figure 9.22 The first stage of the training procedure for EBMs, where models for each feature are trained sequentially and cyclically, with one model per feature per iteration. The trees trained are shallow, and the learning rate is very low. However, over a very large number of iterations, a sufficiently complex nonlinear model for each feature can be learned. A similar procedure is also followed for the second stage of training EBMs, where models for pairwise feature interactions are trained.

The training diagram shows why EBMs can remain interpretable despite many boosting rounds: shallow trees are assigned cyclically to named feature or interaction components instead of forming one opaque ensemble. section 9.5.1

Bank marketing classification with InterpretML

InterpretML provides ExplainableBoostingClassifier for training, visualization, and interpretation. For imbalanced bank-marketing data, positive examples receive weight 5.0 and negative examples weight 1.0. With 19 features, the classifier considers 171 possible pairs but selects the top 10 by default. The reported EBM achieves 86.69% accuracy and 74.59% balanced accuracy, close to the cited XGBoost result, while exposing local contributions and global feature-importance and dependency views. section 9.5.2 section 9.5.2

How Ensemble Methods for Machine Learning covers it

How deep it goes

Explains additive single-feature and pairwise components, cyclic shallow-tree training, and InterpretML use without deriving the boosting objective.

Examples

Uses diabetes diagnosis with age, blood glucose, and BMI, plus weighted Bank Marketing classification and a locally explained test prediction.

What it assumes

Builds on generalized linear models and generalized additive models while introducing pairwise extensions and cyclic boosting conceptually.

What you can do after

Apply via InterpretML to train an ExplainableBoostingClassifier and inspect global dependencies or local component contributions.

How it connects

Connects generalized additive models and generalized linear models with gradient boosting, feature interactions, LIME, SHAP, and XGBoost.

liveBookmanning.com

Read Ensemble Methods for Machine Learning on liveBook Get the book at manning.com

Related concepts

• generalized additive modelkind of · A generalized additive model predicts an outcome by combining separate learned functions of individual features, optionally through a response-link function.
• feature interactionuses · A feature interaction is a joint effect in which a model’s output or behavior depends on a combination of input features rather than on each feature independently.
• XGBoostcontrasts with · XGBoost is an open-source software library for efficient gradient-boosted decision-tree algorithms used in supervised machine learning.
• gradient boostinguses · Gradient boosting is an ensemble-learning technique that sequentially adds predictive models, typically weak learners, to reduce a loss by fitting its gradient.

This explainer was generated by Manning's knowledge graph from the text of Ensemble Methods for Machine Learning. The section links point to the parts of the book it draws on.