Feature interaction

As explained in Ensemble Methods for Machine Learning

A pairwise feature interaction is a joint effect in which a model's output depends on a combination of two features rather than either feature independently.

A feature interaction is a joint effect produced by two features together. Unlike a generalized additive model, where each component depends on one feature, an explainable boosting machine can include a component that depends on a pair. section 9.5.1

The pairwise component captures behavior that neither feature explains independently. Because the model includes only selected pairs, it preserves a compact, interpretable structure instead of expanding to every possible combination. section 9.5.1 section 9.5

An EBM handles feature interactions by:

• Representing predictions as contributions from individual features and selected feature pairs. section 9.5.1
• Searching among possible pairs and retaining only the most important interactions; with 19 features, an example model considers 171 pairs but keeps 10 by default. section 9.5.2
• Training interaction components in a second cyclic stage after learning individual-feature components. section 9.5.1
• Showing each pair's contribution directly in local explanations and dependency-style visualizations. section 9.5.2

Diabetes diagnosis

An EBM using age, blood glucose level, and body mass index could include all three individual components but retain only one pair, such as the most significant interaction among the three possible pairs. In bank-marketing predictions, month × day_of_week is an example of a pairwise feature with a positive contribution for a particular customer. section 9.5.2

Figure 9.21 An EBM is a generalized additive model consisting of nonlinear components that depend on only one feature as well as nonlinear components that depend on pairs of features. This example shows an EBM for diabetes diagnosis dependent on three variables: age, glc, and bmi. Though there are three pairs of variables (age-glc, glc-bmi, age-bmi), this EBM includes only one of them that it has deemed significant. The explainable boosting model is also an ensemble.

Use the figure to compare single-feature components with a selected pairwise component and see how interaction sparsity supports interpretability. section 9.5.1

How Ensemble Methods for Machine Learning covers it

How deep it goes

Defines pairwise components, explains selective inclusion, and connects them to cyclic EBM training and local explanations.

Examples

Uses diabetes variables age, glucose, and BMI, plus bank-marketing pairs such as month × day_of_week.

What it assumes

Assumes familiarity with generalized linear models before extending additive components to feature pairs.

What you can do after

Operate/configure an EBM to select, visualize, and interpret important pairwise contributions.

How it connects

Connects feature interactions with generalized linear models, generalized additive models, explainable boosting machines, SHAP, and LIME.

liveBookmanning.com

Read Ensemble Methods for Machine Learning on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIA feature interaction is a predictive effect produced jointly by two or more input features rather than by each feature independently.
• 

  liveBookmanning.com
  Machine Learning System DesignA dependency or combined effect between features that influences their computation or a model's behavior.
• 

  liveBookmanning.com
  Outlier Detection in PythonA feature interaction is a dependency in which the effect of one feature on a model output varies according to the value of another feature.

Compare how 4 books explain feature interaction →

Related concepts

• Explainable Boosted Machineused by · An Explainable Boosted Machine is an interpretable additive model trained with boosted shallow trees to learn individual feature effects and selected interactions.
• parallelismconfigures · Parallelism is the concurrent execution or progress of multiple computational operations or tasks.
• tree-based methodaddressed by · A tree-based method is a predictive or decision-making technique that represents its rules or decision structure using one or more trees.
• linear regressionuses · Linear regression is a statistical modeling method that estimates a continuous response as a linear combination of predictor variables and fitted coefficients.
• linearly separableaddresses · Linearly separable describes labeled data whose classes can be divided by a single linear decision boundary in feature space.
• gradient boostinguses · Gradient boosting is an ensemble-learning technique that sequentially adds predictive models, typically weak learners, to reduce a loss by fitting its gradient.

This explainer was generated by Manning's knowledge graph from the text of Ensemble Methods for Machine Learning. The section links point to the parts of the book it draws on.