Random forest tree

As explained in Ensemble Methods for Machine Learning

A randomized decision tree is a decision tree whose split construction evaluates a randomly selected subset of features rather than all available features.

A random forest tree is a randomized decision tree: at each decision node, it evaluates a randomly selected subset of features instead of exhaustively checking every feature. This prevents greedy tree learning from repeatedly favoring the same dominant features and increases diversity among trees. section 2.3.1

The split-selection process works as follows:

• Randomly sample a subset of available features before constructing a decision node. section 2.3.1
• Evaluate candidate splits only among those sampled features. section 2.3.1
• Choose the highest-scoring split from that restricted set and continue growing the tree. section 2.3.1
• Combine such trees in a random forest, alongside bootstrap sampling; classification uses majority voting, while regression uses averaged predictions. section 2.3.1 section 7.2.1

Six-feature split

With features {f1, f2, f3, f4, f5, f6}, a standard tree evaluates all six and might choose f3. A randomized tree first samples a subset, then chooses the best split only from the available features. section 2.3.1

Figure 2.7 Random forests use a modified tree learning algorithm, where a random feature subset is first chosen before the best splitting criterion for each decision node is identified. The unshaded columns represent features that have been left out; the lightly shaded columns represent available features from which the best feature is chosen, shown in the darkly shaded columns.

Visualizes the sampled and excluded feature columns used to construct each decision node. section 2.3.1

How Ensemble Methods for Machine Learning covers it

How deep it goes

Explains node-level feature sampling and connects it to bootstrap-based forests, stopping short of split-criterion internals.

Examples

Uses a six-feature split example and a feature-subset diagram to show restricted candidate selection.

What it assumes

Assumes familiarity with decision-tree splitting and explains the added feature-randomization step.

What you can do after

Evaluate how randomized trees diversify forests and apply them through random-forest ensemble estimators.

How it connects

Connects decision trees, bagging, bootstrap sampling, ensemble aggregation, and Extra Trees.

liveBookmanning.com

Read Ensemble Methods for Machine Learning on liveBook Get the book at manning.com

Related concepts

• random forestused by · A random forest is an ensemble model that combines predictions from multiple decision trees built with randomized data samples, feature selections, or both.
• feature selectionuses · Feature selection is the process of choosing a subset of input features for use in a modeling, analysis, or prediction task.
• decision treekind of · A decision tree is a branching structure that represents sequential tests or choices, with paths leading to outcomes, predictions, or actions.
• extremely randomized treesalternative to · Extremely randomized trees are an ensemble of decision trees that randomizes feature selection and candidate split thresholds during construction.

This explainer was generated by Manning's knowledge graph from the text of Ensemble Methods for Machine Learning. The section links point to the parts of the book it draws on.