Information gain measures how much uncertainty a decision-tree split removes. Compare the parent node’s entropy with the weighted entropy of its child leaves: a larger reduction indicates a better split. section 9.18 section 9.6
Use the idea in three steps:
- Measure the parent node’s entropy. A completely uniform label set has entropy 0; a balanced two-label set has entropy 1. section 9.18
- Split the samples and calculate each leaf’s entropy, weighting each result by that leaf’s share of the samples. section 9.3
- Prefer the split with the greatest entropy reduction, or equivalently the lowest weighted entropy when the parent stays fixed. section 9.6
Choosing age over platform
For six app users, splitting by platform gives an average entropy of 0.918. Splitting by age gives 0.459 because all young users share one label. Since age leaves less remaining uncertainty, age provides greater information gain.
Table 9.2 A simplified version of the dataset in table 9.1, where the age column has been simplified to two categories, “young” and “adult”
|
Platform
|
Age
|
App
|
|
iPhone
|
Young
|
Atom Count
|
|
iPhone
|
Adult
|
Check Mate Mate
|
|
Android
|
Adult
|
Beehive Finder
|
|
iPhone
|
Adult
|
Check Mate Mate
|
|
Android
|
Young
|
Atom Count
|
|
Android
|
Young
|
Atom Count
|
Inspect the small platform-and-age dataset behind the entropy comparison, including its app labels. section 9.3






