Data aggregation combines multiple outputs into one result. In ensemble learning, aggregating predictions from different decision trees reduces dependence on any single tree, whose structure may change substantially with its training sample. The individual trees remain unstable; the collective prediction can be more stable. section 9.0 section 9.2.2
In bagging, each tree is trained on a bootstrap sample drawn with replacement from the original observations. For classification, the ensemble predicts the class with the most votes. For regression, it averages the numerical predictions. With B trees, the regression prediction at x is ŷ(x) = (1/B) ∑ᵦ₌₁ᴮ Tᵦ(x), where Tᵦ(x) is tree b’s prediction. Unusually high and low predictions can balance one another. section 9.2.1 section 9.2.2
A vote across trees
If 62 of 100 trees predict one class and 38 predict another, the ensemble selects the class with 62 votes. A single incorrect vote need not determine the result. section 9.2.2 section 9.2.3
Figure 9.1 Bagging with decision trees. Bootstrap sampling creates multiple training sets by drawing observations with replacement, so each sample contains duplicated and omitted observations. A separate decision tree is trained on each sample using the same procedure. The trees’ predictions are then combined by majority vote for classification or averaging for regression, producing a more stable final prediction.

Follow bootstrap samples through separate trees to a vote or average. section 9.2.2
Aggregation works best when trees do not make the same errors. If a strong predictor produces similar early splits across trees, their errors may be correlated, and voting or averaging cannot cancel a mistake shared by most of them. Random forests add random feature selection at each split to encourage diversity; limiting the candidate features can also weaken individual trees, so generalization depends on their strength as well as their correlation. section 9.2.3 section 9.9
Adding trees reduces variability associated with a finite ensemble, but a stable prediction is not necessarily an accurate one. More trees do not correct data leakage, unrepresentative samples, unsuitable settings, or errors shared by the forest’s trees. section 9.4.3



















