A decision tree can change substantially when its training sample changes. Because each split creates a hard partition, a small change near the root can redirect observations into different branches and alter later decisions. Deeper trees are especially fragile as they split smaller groups of observations. section 9.1.1
Pruning or limiting tree growth can reduce sensitivity to sample-specific patterns, but cannot make a single tree independent of its sample. Constraints also risk removing genuine nonlinear relationships or interactions. Random forests take another route: they combine diverse trees so their errors can offset, turning individual instability into greater stability in the aggregate. section 9.1.3 section 9.0
Averaging estimates
One person estimating a cow’s weight may be far from the truth; averaging many independent estimates can offset errors in different directions. section 9.1.3
