Decision tree

As explained in Machine Learning Algorithms in Depth

A decision tree is a hierarchical predictive model that makes decisions by applying feature-based tests at internal nodes.

A decision tree is a hierarchical predictive model that partitions an input space into axis-parallel regions. Each leaf classifies its region by majority vote or predicts its expected value for regression. section 5.6

CART grows a binary tree by repeatedly choosing a feature and threshold that produce the lowest partition cost. For feature j, candidate thresholds are midpoints between observed values. Because optimal partitioning is NP-complete, the construction uses a greedy recursive procedure. section 5.6

Training focuses on balancing purity against generalization:

• Represent the training data as an n-by-d matrix, then search each feature and its candidate thresholds. section 5.6
• Evaluate each split using an impurity objective. The Gini index measures expected classification error within a leaf and becomes zero for a pure leaf. section 5.6
• Limit tree depth while minimizing impurity to reduce underfitting and overfitting; entropy provides an alternative objective. section 5.6
• Inspect the resulting tree for interpretability and feature-importance insight. section 5.6

From-scratch CART classifier

A Python implementation can represent each node with its Gini value, sample counts, class counts, class label, selected feature, threshold, and left and right children. A classifier object stores max_depth and searches for the best split; an Iris-data workflow then evaluates predictions with accuracy. section 5.6

Figure 5.14 CART decision tree algorithm pseudo-code

Use the CART pseudocode to follow recursive tree growth, split selection, and stopping structure before reading the implementation. section 5.6

How Machine Learning Algorithms in Depth covers it

How deep it goes

Derives feature-threshold selection, impurity objectives, and recursive growth, then implements CART while deferring complete optimization details.

Examples

Demonstrates CART with an Iris dataset workflow, accuracy scoring, and a from-scratch node-and-classifier implementation.

What you can do after

Build it from scratch by selecting thresholds, minimizing impurity, growing nodes recursively, and evaluating classification predictions.

How it connects

Connects supervised learning, classification, regression, Gini index, entropy, feature importance, and overfitting.

liveBookmanning.com

Read Machine Learning Algorithms in Depth on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Grokking Machine LearningAn algorithm that recursively splits labeled data using feature tests and stopping conditions to construct a decision tree.
• 

  liveBookmanning.com
  Statistics Every Programmer NeedsA decision tree is a graphical decision-analysis model that represents choices, uncertain events, possible outcomes, probabilities, and payoffs as branching paths.
• 

  liveBookmanning.com
  Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIA decision tree is a predictive model that recursively partitions inputs using feature-based split rules and produces outcomes at terminal leaves.
• 

  liveBookmanning.com
  Ensemble Methods for Machine LearningA decision tree is a predictive model that recursively partitions data according to feature-based decisions, producing predictions at the resulting leaves.
• 

  liveBookmanning.com
  Statistics Slam DunkA decision tree is a predictive model that recursively partitions data using feature-based decision rules.
• 

  liveBookmanning.com
  Outlier Detection in PythonA decision tree is a hierarchical predictive model that recursively partitions observations using feature-based decision rules.
• 

  liveBookmanning.com
  Grokking AI Algorithms, Second EditionA decision tree is a predictive model that recursively applies feature-based tests to partition examples and produce a classification or regression outcome.
• 

  liveBookmanning.com
  Data Without LabelsA tree-structured predictive model that recursively splits data into nodes using decision rules.
• 

  liveBookmanning.com
  Inside AIA decision tree is a hierarchical predictive model that recursively splits data according to attribute-based questions to reach decisions or predictions.
• 

  liveBookmanning.com
  Machine Learning Platform EngineeringA decision tree is a predictive model that recursively partitions input data using feature-based decision rules.
• 

  liveBookmanning.com
  Demand Forecasting Best PracticesA decision tree is a machine-learning model that makes predictions by applying a sequence of feature-based questions and following the resulting branches to a terminal prediction.
• 

  liveBookmanning.com
  Data-Driven CybersecurityA predictive model that recursively partitions data through feature-based decision rules arranged in a tree structure.

Compare how 13 books explain decision tree →

Related concepts

• leaf nodehas part · A leaf node is a node in a tree or hierarchical structure that has no child nodes and cannot be further subdivided.
• random forestpart of · A random forest is an ensemble model that combines predictions from multiple decision trees built with randomized data samples, feature selections, or both.
• language modelkind of · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.
• decision nodehas part · A decision node is a tree node where a decision, test, or action selection determines which successor branch is followed.
• decision stumpincludes · A decision stump is a decision tree of depth one with a single decision node and prediction leaves.
• XGBoostused by · XGBoost is an open-source software library for efficient gradient-boosted decision-tree algorithms used in supervised machine learning.

This explainer was generated by Manning's knowledge graph from the text of Machine Learning Algorithms in Depth. The section links point to the parts of the book it draws on.