Tree-based method

As explained in Machine Learning Algorithms in Depth

A tree-based model makes predictions by recursively partitioning an input space into regions represented by nodes and leaves.

A tree-based model recursively partitions an input space into axis-parallel regions. Each leaf predicts a class by majority vote or a regression value by expectation. Training greedily chooses a feature and threshold that minimize partition cost, because optimal space partitioning is NP-complete. section 5.6

CART training centers on three decisions:

• Represent the data as X with n points and d features, then search feature j and midpoint threshold t among candidate thresholds. section 5.6
• Evaluate partition quality using impurity; the Gini index estimates expected classification error and becomes zero for a pure leaf. section 5.6
• Control tree depth while reducing impurity to limit underfitting and overfitting; entropy provides an alternative objective. section 5.6

From-scratch CART implementation

A Python implementation defines TreeNode and DecisionTreeClassifier classes, stores Gini statistics and child links, and includes a max_depth control. The accompanying workflow loads Iris data, splits training and test sets, and measures accuracy. section 5.6

Figure 5.14 CART decision tree algorithm pseudo-code

Use the CART pseudo-code to follow recursive tree construction and split selection. section 5.6

How Machine Learning Algorithms in Depth covers it

How deep it goes

Derives feature-threshold search and Gini impurity, mentions entropy, and stops short of detailing the full objective equations.

Examples

Demonstrates Python classes for CART and applies the workflow to the Iris dataset with train-test splitting and accuracy scoring.

What you can do after

Build a basic CART classifier from scratch with node state, split search, depth control, and accuracy evaluation.

How it connects

Connects decision trees with CART, classification, regression, Gini impurity, entropy, feature importance, and overfitting.

liveBookmanning.com

Read Machine Learning Algorithms in Depth on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIA tree-based model is a predictive model whose decision structure is formed from one or more decision trees.
• 

  liveBookmanning.com
  Demand Forecasting Best PracticesA tree-based machine learning model makes predictions using one or more decision-tree structures that represent relationships between inputs and outputs.

Compare how 3 books explain tree-based method →

Related concepts

• random forestincludes · A random forest is an ensemble model that combines predictions from multiple decision trees built with randomized data samples, feature selections, or both.
• importance scoreused by · An importance score is a numerical value indicating an item's relative significance, relevance, or contribution within a specified context.
• XGBoostincludes · XGBoost is an open-source software library for efficient gradient-boosted decision-tree algorithms used in supervised machine learning.
• gradient boostingincludes · Gradient boosting is an ensemble-learning technique that sequentially adds predictive models, typically weak learners, to reduce a loss by fitting its gradient.
• decision treeincludes · A decision tree is a branching structure that represents sequential tests or choices, with paths leading to outcomes, predictions, or actions.
• label encodingused by · Label encoding is a categorical-data transformation that maps each category to a numerical code, which may imply an ordering among categories.

This explainer was generated by Manning's knowledge graph from the text of Machine Learning Algorithms in Depth. The section links point to the parts of the book it draws on.