Gini impurity
Gini impurity is a measure of class-label heterogeneity in a set or decision-tree node, calculated from the proportions of its classes.
Grokking Machine Learning
A decision-tree impurity measure that quantifies the heterogeneity of class labels within a node.
- How it is used
- Selects classification-tree questions by favoring splits that leave less-mixed groups of labels.
- How deep it goes
- Derives the probability formula from two random draws and works through child-group averages, including weighting unequal groups.
- Examples
- Uses colored balls, app recommendations, age cutoffs, and a credit-card-fraud exercise to make split comparisons concrete.
- What you can do after
- Build it from scratch by calculating class shares, weighting child-group impurities, and selecting a lower-impurity split.
Statistics Every Programmer Needs
Gini impurity is a node-impurity measure equal to the probability of misclassifying an observation when labels are assigned according to the node's class distribution.
- How it is used
- Ranks candidate classification-tree splits by how well their branches separate class labels.
- How deep it goes
- Works through class counts and size-weighted impurity for categorical and numeric splits, while treating entropy mainly as an alternative.
- Examples
- Uses fourth-down conversion counts grouped by play type and by a distance-to-go threshold.
- What you can do after
- Apply it via scikit-learn classifiers and calculate weighted impurity to compare candidate root splits.
Read how Statistics Every Programmer Needs explains Gini impurity →
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
Gini impurity is the probability of misclassifying an element labeled according to the class distribution of its subset.
- How it is used
- Guides classification-tree split selection by favoring child subsets with more homogeneous target labels.
- How deep it goes
- Defines the misclassification probability and demonstrates threshold scoring, but stops short of a mathematical derivation.
- Examples
- Uses Airbnb price-range prediction, latitude thresholds, exhaustive candidate search, and 256-bin histogram candidates.
- What you can do after
- Build a split-scoring routine from scratch and compare exhaustive threshold search with histogram-based candidates.
Read how Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI explains Gini impurity →
Grokking AI Algorithms, Second Edition
The Gini index is an impurity measure that quantifies the likelihood of incorrect class assignment in a labeled dataset.
- How it is used
- Measures dataset impurity while training decision trees and comparing candidate splits.
- How deep it goes
- Defines impurity, contrasts pure and mixed datasets, and implements the calculation without theoretical proof.
- Examples
- Uses Perfect and Okay diamonds, including balanced and completely pure bags, to illustrate impurity values.
- What you can do after
- Apply it via Python to calculate impurity and compare candidate decision-tree splits.
Read how Grokking AI Algorithms, Second Edition explains Gini impurity →
Machine Learning Algorithms in Depth
The Gini index is an impurity measure for evaluating the heterogeneity of class labels in a set or node.
- How it is used
- Evaluates candidate CART partitions and guides feature-threshold selection toward purer classification leaves.
- How deep it goes
- Explains class proportions, purity, expected error, and a threshold example, while contrasting entropy as an alternative objective.
- Examples
- Uses a five-point one-dimensional dataset, threshold 2, pure left leaf, and mixed right leaf.
- What you can do after
- Evaluate candidate classification splits and implement the surrounding CART search procedure.
Read how Machine Learning Algorithms in Depth explains Gini impurity →
Related concepts
- entropyalternative to · Entropy is a quantitative measure of uncertainty or dispersion in a probability distribution and, more generally, of disorder in the possible states of a system.
- CARTused by · CART is a decision-tree learning algorithm that recursively partitions data into binary branches for classification or regression.
- decision nodemeasures · A decision node is a tree node where a decision, test, or action selection determines which successor branch is followed.
- fairnessmeasures · Fairness is the principle or property of allocating treatment, opportunities, or outcomes equitably among relevant people, groups, or participants according to appropriate criteria.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.