liveBook · Manning

Decision tree node

As explained in Grokking Machine Learning, Second Edition

A tree node is a vertex in a decision tree that represents a split condition or a terminal prediction and may connect to child branches.

A decision tree node is a vertex that asks a yes-or-no question or returns a prediction. The root starts the process, branches carry answers, and leaves finish it. section 9.0

A node can play one of these roles:

  • Root node: the topmost node containing the first question. section 9.0
  • Decision node: a node with two branches that divides data according to a feature-based question. section 9.0 section 9.4
  • Leaf node: a node with no branches that returns a prediction, often the majority label among its samples. section 9.4 section 9.13

From questions to predictions

To decide whether to wear a jacket, start with “Is it raining?” If not, ask whether it is cold. Each answer follows a branch until a leaf says “wear a jacket” or “do not wear a jacket.” A decision stump uses only the first question and two leaves. section 9.0

Figure 9.2 A regular decision tree with a root node, decision nodes, branches, and leaves. Note that each decision node contains a yes-or-no question. From each possible answer, one branch emanates, which can lead to another decision node or a leaf. This tree has a depth of 2, because the longest path from a leaf to the root goes through two branches.

Use the tree diagram to trace questions from the root through branches to leaves and see how depth counts the longest root-to-leaf path. section 9.0

During training, a node is split when a feature improves classification. Stop splitting when the data is pure or when constraints such as maximum depth or minimum sample counts make the node a leaf. Scikit-Learn records entropy, sample count, and label counts at each node. section 9.4 section 9.11 section 9.13

How Grokking Machine Learning, Second Edition covers it

How deep it goes
Defines root, decision, and leaf nodes, then explains splitting, purity, predictions, and stopping constraints.
Examples
Uses jacket decisions, app-download data, student admissions, tree diagrams, and Scikit-Learn classifier outputs.
What it assumes
Assumes only features, labels, and yes-or-no reasoning while teaching tree vocabulary and traversal from zero.
What you can do after
Build a decision tree from scratch or operate Scikit-Learn nodes using splits, predictions, and stopping parameters.
How it connects
Connects nodes with branches, leaves, decision trees, features, predictions, entropy, and Gini index.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning, Second Edition. The section links point to the parts of the book it draws on.