Gini impurity, also called the Gini index, measures impurity rather than uncertainty in bits. It can be interpreted as the probability of misclassification when a node’s class is assigned randomly according to its class distribution. Decision trees use it to compare candidate splits and favor partitions with increasingly homogeneous outcomes. It often selects splits similar to entropy and can be computationally more efficient. section 5.3.1
Gini impurity
Gini impurity measures the probability of misclassifying an observation when its class is assigned randomly according to the class distribution in a node.
How Timeless Algorithms, The Foundational Ideas covers it
- How deep it goes
- Contrasts impurity with entropy, noting similar split preferences and computational simplicity.
- Examples
- Uses loan-approval outcomes to illustrate nodes with balanced or dominant classes in the discussion of homogeneous splits.
- What you can do after
- Evaluate candidate decision-tree splits alongside entropy, expecting similar trees and little difference in predictive performance.
- How it connects
- Connects decision trees, entropy, and information gain through the goal of producing homogeneous groups.
Also explained in
-
Grokking Machine LearningA decision-tree impurity measure that quantifies the heterogeneity of class labels within a node.
-
Grokking Machine Learning, Second EditionThe Gini index is a decision-tree impurity measure based on the probability of misclassification under the label distribution at a node.
-
Statistics Every Programmer NeedsGini impurity is a node-impurity measure equal to the probability of misclassifying an observation when labels are assigned according to the node's class distribution.
-
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIGini impurity is the probability of misclassifying an element labeled according to the class distribution of its subset.
-
Grokking AI Algorithms, Second EditionThe Gini index is an impurity measure that quantifies the likelihood of incorrect class assignment in a labeled dataset.
-
Build AI Drug Discovery PipelinesGini impurity measures the expected misclassification rate when class labels are randomly assigned according to their distribution within a node.
-
Machine Learning Algorithms in DepthThe Gini index is an impurity measure for evaluating the heterogeneity of class labels in a set or node.
-
Fight Fraud with Machine LearningGini impurity is a measure of the class mixture within a decision-tree node, with zero indicating a node containing a single class.
Compare how 9 books explain Gini impurity →
Related concepts
- entropyalternative to · Entropy is an information-theoretic measure of uncertainty in a probability distribution, defined as the expected value of the negative logarithm of outcome probability.
- CARTused by · CART is a decision-tree learning algorithm that recursively partitions data into binary branches for classification or regression.
- decision nodemeasures · A decision node is a tree node where a decision, test, or action selection determines which successor branch is followed.
- fairnessmeasures · Fairness is the equitable allocation of treatment, opportunities, outcomes, or service among relevant people, groups, or tasks according to specified criteria.
- puritymeasures · Purity is a measure of how strongly groups are dominated by a single label, often expressed as the proportion bearing each group’s most common label.
This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.








