Gini impurity

As explained in Grokking AI Algorithms, Second Edition

The Gini index is an impurity measure that quantifies the likelihood of incorrect class assignment in a labeled dataset.

Gini impurity measures how mixed a labeled dataset is. A value of 0 means every example has the same class; larger values indicate greater uncertainty. Decision trees use it to judge whether a proposed question makes the data cleaner. section 8.17

A bag of diamonds

A bag containing five Perfect and five Okay diamonds is maximally mixed, with a Gini index of 0.5. A bag containing ten Perfect diamonds has a Gini index of 0. After testing feature questions, the tree compares the weighted impurity of the resulting groups with the original impurity; the reduction is information gain. section 8.17

Figure 8.21 The Gini index calculation

Use the calculation figure to connect class proportions with the impurity score. section 8.17

How Grokking AI Algorithms, Second Edition covers it

How deep it goes

Defines impurity, contrasts pure and mixed datasets, and implements the calculation without theoretical proof.

Examples

Uses Perfect and Okay diamonds, including balanced and completely pure bags, to illustrate impurity values.

What it assumes

Assumes labeled examples and basic feature-based classification, while introducing impurity and uncertainty through concrete examples.

What you can do after

Apply it via Python to calculate impurity and compare candidate decision-tree splits.

How it connects

Connects decision trees with entropy, information gain, classification labels, and feature-based data splits.

liveBookmanning.com

Read Grokking AI Algorithms, Second Edition on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Grokking Machine LearningA decision-tree impurity measure that quantifies the heterogeneity of class labels within a node.
• 

  liveBookmanning.com
  Statistics Every Programmer NeedsGini impurity is a node-impurity measure equal to the probability of misclassifying an observation when labels are assigned according to the node's class distribution.
• 

  liveBookmanning.com
  Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIGini impurity is the probability of misclassifying an element labeled according to the class distribution of its subset.
• 

  liveBookmanning.com
  Machine Learning Algorithms in DepthThe Gini index is an impurity measure for evaluating the heterogeneity of class labels in a set or node.

Compare how 5 books explain Gini impurity →

Related concepts

• entropyalternative to · Entropy is a quantitative measure of uncertainty or dispersion in a probability distribution and, more generally, of disorder in the possible states of a system.
• CARTused by · CART is a decision-tree learning algorithm that recursively partitions data into binary branches for classification or regression.
• decision nodemeasures · A decision node is a tree node where a decision, test, or action selection determines which successor branch is followed.
• fairnessmeasures · Fairness is the principle or property of allocating treatment, opportunities, or outcomes equitably among relevant people, groups, or participants according to appropriate criteria.

This explainer was generated by Manning's knowledge graph from the text of Grokking AI Algorithms, Second Edition. The section links point to the parts of the book it draws on.