A decision node is a question inside a decision tree. Like a fork in a path, it sends examples one way when the answer is true and another way when it is false. A leaf node ends the path with a classification. section 8.17
Sorting diamonds by cut
Suppose the goal is to classify diamonds as Okay or Perfect using their carat and price. Ask, “Is Carat ≥ 1.13?” In the sample data, all four diamonds answering true are Perfect. The false branch still contains both labels, so it needs another question. “Is Carat ≥ 0.76?” separates three Okay diamonds from the smaller diamonds, which need further sorting. These are illustrative questions, not necessarily the questions a training algorithm would choose. section 8.16 section 8.17
To build the tree, try possible questions at each point and compare how much they reduce uncertainty. A question combines a feature, a value, and—in this simple implementation—a ≥ comparison. The selected decision node holds that question and two branches; the process repeats on each branch until it creates leaves. Constraints matter because a tree can memorize its training examples instead of finding patterns that generalize. section 8.17 section 8.16
class ExamplesNode: #A
def __init__(self, examples):
self.predictions = find_unique_label_counts(examples)
def __repr__(self):
return f"Leaf: {self.predictions}"
class DecisionNode: #B
def __init__(self, question, true_branch, false_branch):
self.question = question
self.true_branch = true_branch
self.false_branch = false_branch
def __repr__(self):
return f"DecisionNode: {self.question}"
def build_tree(examples, feature_names):
gain, question = find_best_split(examples, feature_names)
if gain == 0:
return ExamplesNode(examples) #C
true_examples, false_examples = split_examples(examples, question) #D
true_branch = build_tree(true_examples, feature_names) #E
false_branch = build_tree(false_examples, feature_names) #E
return DecisionNode(question, true_branch, false_branch)
The Python listing represents a decision node with a question, a true branch, and a false branch. Its tree-building function splits examples, builds both branches recursively, and returns a leaf when it finds no gain. section 8.17
