liveBook · Manning

Weighted average Gini impurity

As explained in Grokking Machine Learning

A split-quality measure equal to the size-weighted average of the Gini impurity indices of the resulting subsets.

When a decision-tree question divides data into subsets, weighted average Gini impurity measures how mixed the resulting groups are. It averages each subset’s Gini impurity according to that subset’s size, so lower values indicate cleaner splits. section 9.3

Splitting app users by age

The cutoff 14.5 creates {A, A}, which is pure and has Gini impurity 0, and {A, C, B, C}, whose label proportions are 1/4, 1/4, and 1/2. Combining these impurities by subset size gives a weighted index of 0.417. The cutoff 20 is better in this dataset: it has the lowest weighted index, 0.222, and the highest accuracy, 5/6. section 9.3

How Grokking Machine Learning covers it

How deep it goes
Calculates impurity for concrete subsets and compares seven candidate cutoffs, without extending beyond the split-selection example.
Examples
Uses an app-recommendation dataset containing user ages and labels, with seven possible age-based splits.
What it assumes
Assumes familiarity with decision-tree splits and class labels while teaching purity through explicit small-set calculations.
What you can do after
Evaluate candidate decision-tree questions and select the split with the lowest weighted impurity in the demonstrated dataset.
How it connects
Contrasts with accuracy and entropy while supporting decision-tree splitting and Gini impurity analysis.
Cover of Grokking Machine Learning
liveBookmanning.com
Read Grokking Machine Learning on liveBook Get the book at manning.com

Also explained in

Compare how 2 books explain weighted average Gini impurity →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning. The section links point to the parts of the book it draws on.