How deeply does Ensemble Methods for Machine Learning cover exclusive feature bundling?

Defines feature bundling’s purpose and contrast with GOSS but stops short of detailing the bundling algorithm or implementation.

Exclusive feature bundling

As explained in Ensemble Methods for Machine Learning

Feature bundling is a tree-learning optimization that groups compatible features to reduce the effective number of features processed.

Exclusive feature bundling (EFB) speeds tree learning on high-dimensional data by grouping compatible features, reducing the effective number of features processed during training. section 5.3 section 5.3.1

EFB addresses a specific bottleneck in histogram-based gradient boosting:

Histogram-bin construction can become slow when a dataset contains many observations, many features, or both. section 5.3.1
Instead of reducing training examples, as Gradient-based One-Side Sampling does, EFB reduces the number of features processed. section 5.3.1
The resulting reduction can improve training speed and lower memory usage while supporting large-scale datasets. section 5.3

EFB is one of LightGBM’s algorithmic speedups alongside histogram-based learning and GOSS; it targets feature dimensionality rather than sample count. section 5.3 section 5.6

How Ensemble Methods for Machine Learning covers it

Cover of Ensemble Methods for Machine Learning

liveBook manning.com

Read Ensemble Methods for Machine Learning on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIExclusive feature bundling is a feature-dimensionality-reduction technique that groups sparse, mutually exclusive features while preserving their nonzero information.

Compare how 2 books explain exclusive feature bundling →

Related concepts

LightGBMused by · LightGBM is an open-source gradient-boosting framework for efficiently training tree-based machine-learning models.
one-hot encodingcontrasts with · One-hot encoding represents each category as a binary vector or set of indicators with exactly one active component identifying that category.
dimensionality reductionimplements · Dimensionality reduction is the transformation of data into a representation with fewer dimensions while retaining selected information, structure, or relationships.
greedy algorithmimplemented by · A greedy algorithm builds a solution incrementally by choosing the locally best available option at each step, without generally accounting for all future consequences or revising prior choices.

This explainer was generated by Manning's knowledge graph from the text of Ensemble Methods for Machine Learning. The section links point to the parts of the book it draws on.