Model complexity measures a model’s flexibility: its function class may contain more parameters, features, or nonlinear transformations. Greater complexity increases capacity to learn varied relationships, but does not guarantee better predictions. section 4.4.1 section 3.1.1
In drug discovery, complexity matters when predicting properties such as solubility or classifying compounds as hERG blockers. A flexible model can capture complex molecular patterns, provided sufficient repeated data supports generalization to similar unseen molecules. section 3.1.1 section 4.0
Increasing or decreasing complexity creates a bias–variance trade-off:
- Low complexity can impose overly simple assumptions, producing high bias, underfitting, and high training error. section 4.4.1 section 4.4.3
- High complexity can fit training noise, producing high variance and performance that fails to generalize to unseen data. section 4.4.1 section 4.4.3
- An intermediate range often minimizes test error by balancing insufficient flexibility against excessive sensitivity to the training set. section 4.4.3
Polynomial fits
A second-order polynomial underfits data sampled from a noisy seventh-order target function, while a twentieth-order polynomial can overfit its noise. The simpler model nevertheless achieves lower test error in the comparison. section 4.4.1
Validation curves help select complexity by plotting training and test performance across hyperparameter values. In SVR, L2 regularization, polynomial feature degree, and epsilon-tube width change effective complexity: stronger regularization and wider margins simplify the model, while higher polynomial degree increases nonlinearity. section 4.4.3
Figure 4.17 Impact of model complexity on the fitted model’s training and testing performance.

Compare how low- and high-complexity polynomial models affect fitted curves, training performance, and testing performance. section 4.4.1
Figure 4.20 Validation curves for SVR and preprocessing hyperparameters.

Inspect how SVR regularization, polynomial degree, and error-margin settings alter complexity and generalization. section 4.4.3



