Noise is stochastic variation in observations that systematic model structure does not explain. In regression, it appears as the error term separating observed responses from the fitted line. section 4.1.1
Noise matters because it changes how confidently a model can learn and generalize:
- High noise can limit a model’s ability to distinguish meaningful patterns from irrelevant variation. section 6.4.3
- Noise contributes to imperfect predictions even when a regression line captures the systematic relationship between variables. section 4.1.1
- Random forests may reduce overfitting and generalize more robustly than single trees, but noise can still constrain predictive performance. section 6.4.3
Regression observations around a fitted line
A scatter plot can show observations distributed around a regression line. The vertical difference between an observed response and its predicted value represents variation not captured by the fitted relationship. section 4.1.1
