A random forest combines many decision trees into one supervised model. For classification, the trees vote for a class; for regression, their predictions are averaged. Aggregating trees can improve stability and reduce overfitting compared with relying on a single tree, though the resulting model is harder to interpret. section 6.0 section 6.4
Each tree trains on a sample drawn from the training data with replacement—a process called bagging. At each split, the forest also considers a random subset of features. These two sources of variation make the trees less alike, so the combined prediction is less dependent on any one tree's decisions. section 6.1
A practical classification workflow is:
- Separate training and test data, then fit a forest on the training features and labels. Set the number of trees and, where useful, controls such as the split criterion and maximum depth. section 6.4.1
- Predict labels for the held-out test features. Compare them with the true labels using accuracy and a confusion matrix; inspect class-level errors rather than relying on accuracy alone. section 6.4.2 section 6.4.3
- Interpret feature importance across the forest, or inspect a few constituent trees to see how their features and splits vary. Neither view makes the entire ensemble as readable as one decision tree. section 6.5 section 6.4.5
>>> from sklearn.ensemble import RandomForestClassifier >>> rf = RandomForestClassifier(n_estimators = 50, >>> criterion = 'gini', >>> max_depth = 3, >>> random_state = 0)
This scikit-learn configuration uses 50 trees, Gini impurity for splits, a maximum tree depth of three, and a fixed random state. Fit it with `rf.fit(X_train, y_train)` before calling `rf.predict(X_test)`. More trees may help, but gains can diminish while computation grows. section 6.4.1 section 6.4.2
A modest accuracy gain
On a held-out classification test set, a random forest reached 62% accuracy versus 61% for a decision tree. The gain is real for this comparison but small: noisy or weak predictors, class imbalance, outliers, or untuned settings may limit performance. Use the confusion matrix to investigate which classes the model gets wrong before deciding that the forest is preferable. section 6.4.3
Random forests are useful when predictive accuracy and resilience to noisy data matter more than a simple decision path. Their diversity does not guarantee a large improvement on every data set, and larger forests cost more to fit. section 6.4 section 6.4.1
