Node impurity
Node impurity is a measure of the heterogeneity of class labels or outcomes among observations in a decision-tree node.
Machine Learning Algorithms in Depth
Impurity is a measure of how mixed the class labels are within a data region or tree node.
- How it is used
- Guides CART toward class-homogeneous leaves while controlling tree depth.
- How deep it goes
- Defines Gini and entropy, explains their purity interpretation, and works through one threshold without deriving all implementation details.
- Examples
- Uses a five-point one-dimensional dataset, threshold 2, pure and mixed leaves, and a from-scratch CART implementation.
- What you can do after
- Build it from scratch by evaluating candidate thresholds with Gini-based node costs.
Read how Machine Learning Algorithms in Depth explains node impurity →
Timeless Algorithms, The Foundational Ideas
Impurity is a measure of how mixed the outcomes or classes are within a decision-tree node.
- How it is used
- Uses impurity reduction to select classification splits that separate class labels.
- How deep it goes
- Defines impurity through split selection, naming entropy and information gain without giving a formula.
- What you can do after
- Understand how classification splits reduce uncertainty and regression splits reduce outcome variation.
Read how Timeless Algorithms, The Foundational Ideas explains node impurity →
Also covered in
What to know first
- decision treeA decision tree is a tree-structured model or representation in which successive tests or choices lead through branches to outcomes, predictions, or actions.
- linear regressionLinear regression is a statistical method that models a numeric response as a linear function of one or more predictor variables with fitted parameters.
- overfittingOverfitting is a modeling condition in which a model adapts too closely to particular data, including its noise or idiosyncrasies, and consequently generalizes poorly to new data.
Related concepts
- feature importance scoremeasured by · A feature importance score is a numerical estimate of an input feature’s relative contribution or influence on a model’s predictions or performance.
- decision treeuses · A decision tree is a tree-structured model or representation in which successive tests or choices lead through branches to outcomes, predictions, or actions.
Questions about node impurity
What is node impurity?
Node impurity is a measure of the heterogeneity of class labels or outcomes among observations in a decision-tree node.
Which Manning books explain node impurity?
Machine Learning Algorithms in Depth and Timeless Algorithms, The Foundational Ideas.
What should I know before learning node impurity?
Manning books that teach node impurity usually introduce decision tree, linear regression and overfitting first.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.

