Random forest

As explained in Grokking Machine Learning

An ensemble model that combines predictions from many decision trees, typically trained with randomized data or feature selection.

A random forest combines predictions from multiple decision trees. Each tree is trained on a random subset of the data, and the trees vote on a classification prediction. Combining learners can help reduce overfitting compared with relying on one model. section 12.1 section 12.2

A simple way to picture the process:

• Choose random subsets of the dataset; some examples may appear in several subsets, others in none. section 12.2
• Train a decision tree on each subset. section 12.2
• Combine the trees’ predictions by voting. section 12.2

Classifying email

For a small spam-and-ham dataset, the features are counts of “lottery” and “sale” in each email. A single deep tree fits the training data perfectly but overfits outliers; a forest combines trees to classify the emails. section 12.2

Figure 12.6 The way to obtain the predictions of the random forest is by combining the predictions of the three weak learners. On the top, we can see the three boundaries of the decision trees from figure 12.5. On the bottom, we can see how the three decision trees vote to obtain the boundary of the corresponding random forest.

Compare the individual tree boundaries with the forest’s combined voting boundary. section 12.2

For hands-on use, Scikit-Learn’s RandomForestClassifier fits a forest. Random forests also work for regression; the worked example focuses on classification. section 12.2

How Grokking Machine Learning covers it

How deep it goes

Works through random data subsets, tree training, and voting, then demonstrates a Scikit-Learn classifier.

Examples

Uses spam-versus-ham email features and illustrates a forest voting over individual tree boundaries.

What it assumes

Builds on decision trees, treated as previously learned rather than reintroduced in the forest example.

What you can do after

Apply it via Scikit-Learn’s RandomForestClassifier to fit a classification model.

How it connects

Connects decision trees and bagging with ensemble learning, and contrasts forests with boosting methods.

liveBookmanning.com

Read Grokking Machine Learning on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Statistics Every Programmer NeedsA random forest is an ensemble model that combines predictions from multiple randomized decision trees.
• 

  liveBookmanning.com
  Ensemble Methods for Machine LearningRandomForestRegressor is a scikit-learn estimator that predicts continuous targets by aggregating randomized decision-tree regressors.
• 

  liveBookmanning.com
  Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIAn ensemble of randomized decision trees whose predictions are aggregated.
• 

  liveBookmanning.com
  Outlier Detection in PythonRandom forest is an ensemble learning method that combines predictions from multiple decision trees built using randomized data and feature selection.
• 

  liveBookmanning.com
  Cyber Threat HuntingA random forest is an ensemble machine-learning algorithm that combines predictions from multiple decision trees trained with randomized sampling or feature selection.
• 

  liveBookmanning.com
  Investing for ProgrammersA random forest is an ensemble algorithm that combines predictions from multiple decision trees.
• 

  liveBookmanning.com
  Graph Algorithms for Data ScienceA random forest model is an ensemble of decision trees whose aggregated predictions perform classification or regression.
• 

  liveBookmanning.com
  Data-Driven CybersecurityA random forest is an ensemble machine-learning model that combines predictions from multiple decision trees.
• 

  liveBookmanning.com
  Machine Learning Platform EngineeringA random forest is an ensemble model that combines predictions from multiple decision trees trained with randomized sampling or feature selection.
• 

  liveBookmanning.com
  Financial AI in PracticeA random forest is an ensemble model that combines predictions from multiple decision trees trained with randomized data and feature selection.

Compare how 11 books explain random forest →

Related concepts

• decision treehas part · A decision tree is a branching structure that represents sequential tests or choices, with paths leading to outcomes, predictions, or actions.
• ensemblekind of · An ensemble is a technique that combines or considers outputs from multiple models or component methods collectively to produce or assess a joint result.
• bagginguses · Bagging, or bootstrap aggregation, is an ensemble technique that trains multiple models on bootstrap samples of data and combines their predictions.
• importance scoreuses · An importance score is a numerical value indicating an item's relative significance, relevance, or contribution within a specified context.
• bootstrap samplingused by · Bootstrap sampling is a resampling technique that repeatedly draws observations with replacement from observed data to estimate a statistic’s sampling behavior or variability.

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning. The section links point to the parts of the book it draws on.