liveBook · Manning

Decision stump

As explained in Grokking Machine Learning, Second Edition

A decision stump is a decision tree consisting of a single feature-based question at the root and prediction leaves beneath it.

A decision stump is the smallest decision tree: one yes-or-no question at the root, followed immediately by two leaves. Each answer takes one branch and produces a prediction, so the stump has depth 1. It is a useful first model because its entire decision can be read as one simple question. section 9.0

Figure 9.4 A simple decision tree (decision stump) that consists of only the question, “Is it raining?” If the answer is yes, the decision we make is to wear a jacket. If the answer is no, the decision we make is not to wear a jacket.

A single question—“Is it raining?”—can lead directly to either “wear a jacket” or “do not wear a jacket.” That is the full shape of a decision stump. section 9.0

For classification, a stump asks about one feature and assigns a class at each leaf. The leaf prediction is the most common label among the data points that reached it. The important choice is therefore the root question: compare possible feature splits and keep the one whose two groups best separate the labels. section 9.3

Choosing a question for app recommendations

Suppose users are described by platform and age, and the goal is to predict an app they downloaded. A platform split correctly classifies four of six users when each leaf chooses its most common app. An age split correctly classifies five of six, because all young users downloaded the same app. By accuracy, age is the better stump question. section 9.3

Several measures can compare candidate classification questions:

  • Accuracy counts the fraction of data points classified correctly. section 9.3
  • Gini impurity measures how mixed the labels are in a group; a lower average Gini index favors a cleaner split. section 9.3
  • Entropy also measures label mixing; a lower average entropy favors a cleaner split. section 9.3

A stump can also make numerical predictions. Instead of choosing a class at each leaf, it predicts the average label of the data points in that branch. For a numerical feature such as age, test possible cutoffs, calculate the mean square error for the two resulting predictions, and choose the cutoff with the smallest error. section 9.14

Predicting app engagement from age

With age as the question, “Are you younger than 35?” divides eight users into two groups. The younger group receives its average engagement prediction, and the older group receives its own average. Among the tested age cutoffs, 35 gives the smallest mean square error, so it supplies the stump’s root question. section 9.14

A larger decision tree grows by splitting a leaf again. A stump stops before that second question, making it easy to inspect but limited to one feature-based decision on any prediction path. Practicing with fraud and symptom datasets helps turn the idea into a concrete classifier-building task. section 9.0 section 9.19

How Grokking Machine Learning, Second Edition covers it

How deep it goes
Defines stump structure, compares classification splits with three criteria, and works through regression cutoffs and mean square error.
Examples
Uses jacket decisions, app recommendations by platform or age, COVID-19 symptoms, fraud transactions, and age-based app engagement.
What it assumes
Introduces binary-tree vocabulary from zero and recaps accuracy before comparing candidate splits.
What you can do after
Build it from scratch by selecting a one-question classifier using accuracy, Gini impurity, or entropy.
How it connects
Connects decision trees with root nodes, leaves, accuracy, Gini impurity, entropy, mean square error, classification, and regression.

Also explained in

Compare how 3 books explain decision stump →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning, Second Edition. The section links point to the parts of the book it draws on.