When a decision tree splits data, each resulting subset may have a different mixture of labels. Weighted average Gini impurity combines those subset impurities while giving larger subsets more influence. section 9.9
The goal is to find a split with a small weighted value. A low score means the resulting leaves are relatively pure, so their predictions are more focused. section 9.9 section 9.6
Comparing age-based splits
For the split “Is the user younger than 14.5?”, the two groups contain ages 12 and 14 with labels {A, A}, and ages 15, 25, 32, and 35 with labels {A, C, B, C}. The first group is pure, while the second is mixed; their size-weighted Gini score is 0.417. The split at age 20 performs better, with a score of 0.222 and accuracy of 5/6. section 9.9
Table 9.4 The seven possible questions we can pick, each with the corresponding splitting. In the first set, we put the users who are younger than the cutoff, and in the second set, those who are older than the cutoff.
|
Question
|
First set (yes)
|
Second set (no)
|
Labels
|
Weighted accuracy
|
Weighted Gini impurity index
|
Weighted entropy
|
|
Is the user younger than 7?
|
empty
|
12, 14, 15, 25, 32, 35
|
{},
{A,A,A,C,B,C}
|
3/6
|
0.611
|
1.459
|
|
Is the user younger than 13?
|
12
|
14, 15, 25, 32, 35
|
{A},
{A,A,C,B,C}
|
3/6
|
0.533
|
1.268
|
|
Is the user younger than 14.5?
|
12, 14
|
15, 25, 32, 35
|
{A,A}
{A,C,B,C}
|
4/6
|
0.417
|
1.0
|
|
Is the user younger than 20?
|
12, 14, 15
|
25, 32, 35
|
{A,A,A},
{C,B,C}
|
5/6
|
0.222
|
0.459
|
|
Is the user younger than 28.5?
|
12, 14, 15, 25
|
32, 35
|
{A,A,A,C},
{B,C}
|
4/6
|
0.416
|
0.874
|
|
Is the user younger than 33.5?
|
12, 14, 15, 25, 32
|
35
|
{A,A,A,C,B},
{C}
|
4/6
|
0.467
|
1.145
|
|
Is the user younger than 100?
|
12, 14, 15, 25, 32, 35
|
empty
|
{A,A,A,C,B,C},
{}
|
3/6
|
0.611
|
1/459
|
Compare all seven possible age questions, their resulting label groups, weighted accuracies, weighted Gini impurity indexes, and weighted entropies. section 9.9


