A decision stump is the simplest decision tree: one root decision node asks a single yes-or-no question, then two leaves immediately provide predictions. Its depth is 1. The question can test a category, such as platform or age group, or compare a numerical feature with a cutoff. section 9.0
A stump follows a short prediction process:
- Choose one feature-based question or numerical cutoff for the root node. section 9.0 section 9.6
- Send each example down the yes or no branch. section 9.0
- Assign each leaf a prediction, such as its most common class or its average numerical label. section 9.2 section 9.6
- Compare candidate splits using accuracy, Gini impurity, or entropy for classification; use mean square error for regression. section 9.2 section 9.6
Choosing a useful question
Six users are grouped by platform or age to recommend an app. Splitting by platform classifies four of six users correctly; splitting by age classifies five of six because all three young users downloaded Atom Count. Gini impurity and entropy also favor the age split. section 9.2
A regression stump
For user age and weekly app engagement, test possible cutoffs and predict each side with its average label. A cutoff of 35 produces the smallest listed mean square error, 1.983, so it becomes the root split. section 9.6
Figure 9.4 A simple decision tree (decision stump) that consists of only the question, “Is it raining?” If the answer is yes, the decision we make is to wear a jacket.
Visualizes a stump as the single question “Is it raining?” with an immediate jacket prediction on one branch. section 9.0
