At a decision-tree node, impurity measures how mixed the outcomes are. For classification, a split can be chosen by how much it reduces impurity; entropy is one common measure, and its reduction is information gain. Regression trees instead seek splits that reduce variation in the numerical outcome. section 9.1 section 9.1
Node impurity
Impurity is a measure of how mixed the outcomes or classes are within a decision-tree node.
How Timeless Algorithms, The Foundational Ideas covers it
- How deep it goes
- Defines impurity through split selection, naming entropy and information gain without giving a formula.
- What it assumes
- Builds on entropy as a measure of uncertainty and applies it to classification-tree splits.
- What you can do after
- Understand how classification splits reduce uncertainty and regression splits reduce outcome variation.
- How it connects
- Connects classification impurity to entropy and information gain, and regression splits to outcome variation.
Also explained in
-
Machine Learning Algorithms in DepthImpurity is a measure of how mixed the class labels are within a data region or tree node.
Compare how 2 books explain node impurity →
Related concepts
- feature importance scoremeasured by · A feature importance score is a numerical estimate of an input feature’s relative contribution or influence on a model’s predictions or performance.
- decision treeuses · A decision tree is a tree-structured model or representation in which successive tests or choices lead through branches to outcomes, predictions, or actions.
This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.

