Impurity-based importance summarizes how much a feature contributes to improved splits across a forest. Unlike permutation importance, which measures the increase in prediction error after disrupting a feature, it derives importance from the trees’ splits. section 9.6.1 section 9.9
The score has important limitations. Features with many possible split points—such as continuous variables or categorical variables with many levels—have more chances to produce favorable reductions in impurity or variance. Because the score comes from the data used to train the trees, it can also reward sample-specific patterns. It measures predictive reliance, not causality, and does not show whether a feature raises or lowers predictions. section 9.6.2 section 9.6.2
