Statistical bias is systematic error from simplifying assumptions that prevent a model from capturing the data’s true structure. A high-bias model is too simple, underfits, and usually has relatively high training error. The goal is sufficient complexity to capture real patterns without fitting noise. section 4.4 section 4.4.1
Use bias alongside variance to interpret model behavior:
- Compare training and test errors: high bias commonly produces high training error, while high variance produces a large gap between training and unseen-data performance. section 4.4.1
- Inspect learning curves: if errors converge at a high level, collecting more data is unlikely to solve the problem; a more complex model may help. section 4.4.2
- Inspect validation curves: low-complexity hyperparameters create high bias, while excessive complexity creates high variance; choose the region with minimum test error. section 4.4.3
- Control the trade-off with regularization, which can lower variance while increasing bias; decision-tree limits and pruning provide concrete examples. section 4.4.2 section 5.3.1
Polynomial models for solubility prediction
A second-order polynomial may have high bias because it cannot represent a more complex target function, so it underfits. A twentieth-order polynomial has lower bias but can fit noise and generalize poorly because its variance is high. The simpler model can nevertheless achieve lower test error, showing why complexity alone does not determine performance. section 4.4.1
Figure 4.18 How high bias or high variance impact a model’s sensitivity to the training data it learns from.

Use the figure to compare how high-bias and high-variance fits respond to changes in the training data. section 4.4.1





































