Gini impurity measures how mixed the class labels are at a decision-tree node. It is the probability of an incorrect label if an observation is labeled at random according to that node’s class distribution. A pure node has impurity 0; for two equally represented classes, impurity is 0.5. section 6.3.6
For class proportions p₁, p₂, …, calculate Gini impurity as 1 − Σpᵢ². To assess a proposed split, calculate impurity on each side, then average the two results weighted by their numbers of observations. Prefer the candidate with the lower weighted impurity: comparing unweighted values would disregard how many observations each branch contains. section 6.3.6
Compare fourth-down conversion splits
A split at distance to go ≤ 3.5 produces branches with conversion-label counts (127, 268) and (156, 67). Their Gini impurities are approximately 0.436 and 0.420; weighting by branch sizes 395 and 223 gives approximately 0.431. A play-type split, using counts (64, 175) and (219, 160), gives approximately 0.451. Among these two candidates, distance to go better separates conversion outcomes; selecting the root still requires checking the other candidate splits. section 6.3.6
>>> to_go_less_than_3_5 = \
>>> train_data[train_data['TO_GO'] <= 3.5] #1
>>> to_go_greater_than_3_5 = \
>>> train_data[train_data['TO_GO'] > 3.5] #2
>>> counts_less_than_3_5 = (to_go_less_than_3_5 \
>>> .groupby('CONVERT').size()) #3
>>> print('Counts for TO_GO <= 3.5:') #4
>>> print(counts_less_than_3_5) #5
>>> counts_greater_than_3_5 = (to_go_greater_than_3_5 \
>>> .groupby('CONVERT').size()) #6
>>> print('Counts for TO_GO > 3.5:') #7
>>> print(counts_greater_than_3_5) #8
Counts for TO_GO <= 3.5:
CONVERT
0 127
1 268
dtype: int64
Counts for TO_GO > 3.5:
CONVERT
0 156
1 67
dtype: int64
The pandas grouping code obtains class counts on both sides of the distance-to-go threshold—the inputs needed for the weighted calculation. section 6.3.6
In scikit-learn, `criterion='gini'` selects this split criterion for `DecisionTreeClassifier`; it is also available in `RandomForestClassifier`. Entropy is another classification criterion and can produce different splits. Maximum depth is a separate setting that limits tree growth to help control overfitting. section 6.3.2 section 6.4.1
