liveBook · Manning

Node impurity

As explained in Timeless Algorithms, The Foundational Ideas

Impurity is a measure of how mixed the outcomes or classes are within a decision-tree node.

At a decision-tree node, impurity measures how mixed the outcomes are. For classification, a split can be chosen by how much it reduces impurity; entropy is one common measure, and its reduction is information gain. Regression trees instead seek splits that reduce variation in the numerical outcome. section 9.1 section 9.1

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Defines impurity through split selection, naming entropy and information gain without giving a formula.
What it assumes
Builds on entropy as a measure of uncertainty and applies it to classification-tree splits.
What you can do after
Understand how classification splits reduce uncertainty and regression splits reduce outcome variation.
How it connects
Connects classification impurity to entropy and information gain, and regression splits to outcome variation.

Also explained in

Compare how 2 books explain node impurity →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.