liveBook · Manning

Stability

As explained in Timeless Algorithms, The Foundational Ideas

Stability is the persistence of patterns and relationships in data under comparable conditions or over time.

A decision tree can change substantially when its training sample changes. Because each split creates a hard partition, a small change near the root can redirect observations into different branches and alter later decisions. Deeper trees are especially fragile as they split smaller groups of observations. section 9.1.1

Pruning or limiting tree growth can reduce sensitivity to sample-specific patterns, but cannot make a single tree independent of its sample. Constraints also risk removing genuine nonlinear relationships or interactions. Random forests take another route: they combine diverse trees so their errors can offset, turning individual instability into greater stability in the aggregate. section 9.1.3 section 9.0

Averaging estimates

One person estimating a cow’s weight may be far from the truth; averaging many independent estimates can offset errors in different directions. section 9.1.3

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Traces instability from early splits through later branches, then explains why pruning reduces but does not eliminate it.
Examples
Uses averaged cow-weight estimates to illustrate how diverse errors can offset.
What you can do after
Evaluate how sample changes, tree constraints, and aggregation affect prediction reliability on unseen observations.
How it connects
Links decision trees and pruning to random forests and generalization.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.