A deep decision tree can fit noise as well as signal: its local partitions make it expressive, but sensitive to the training sample. Pruning limits this complexity to manage the trade-off between predictive flexibility and stability. section 9.1.2 section 9.1.3
Tree complexity can be limited in two ways:
- Constrain growth by setting a maximum depth, requiring a minimum number of observations in a node or leaf, or demanding a minimum improvement for another split. section 9.1.3
- Grow a large tree first, then remove branches whose added complexity does not produce sufficient improvement. section 9.1.3
A smaller tree may generalize better, vary less across samples, and be easier to interpret. But pruning does not remove the instability of relying on one sample and one tree: different samples can still produce different structures. Restricting growth can also hide genuine nonlinear relationships or interactions. section 9.1.3 section 9.1.2
Pruning manages a single tree’s complexity; it does not resolve the tension between expressiveness and stability. Bagging and random forests offer a different response by aggregating trees rather than asking one constrained tree to represent the whole learning problem. section 9.1.3










