liveBook · Manning

Model complexity

As explained in Build AI Drug Discovery Pipelines

Model complexity is the degree of flexibility or representational richness of a model's function class.

Model complexity measures a model’s flexibility: its function class may contain more parameters, features, or nonlinear transformations. Greater complexity increases capacity to learn varied relationships, but does not guarantee better predictions. section 4.4.1 section 3.1.1

In drug discovery, complexity matters when predicting properties such as solubility or classifying compounds as hERG blockers. A flexible model can capture complex molecular patterns, provided sufficient repeated data supports generalization to similar unseen molecules. section 3.1.1 section 4.0

Increasing or decreasing complexity creates a bias–variance trade-off:

  • Low complexity can impose overly simple assumptions, producing high bias, underfitting, and high training error. section 4.4.1 section 4.4.3
  • High complexity can fit training noise, producing high variance and performance that fails to generalize to unseen data. section 4.4.1 section 4.4.3
  • An intermediate range often minimizes test error by balancing insufficient flexibility against excessive sensitivity to the training set. section 4.4.3

Polynomial fits

A second-order polynomial underfits data sampled from a noisy seventh-order target function, while a twentieth-order polynomial can overfit its noise. The simpler model nevertheless achieves lower test error in the comparison. section 4.4.1

Validation curves help select complexity by plotting training and test performance across hyperparameter values. In SVR, L2 regularization, polynomial feature degree, and epsilon-tube width change effective complexity: stronger regularization and wider margins simplify the model, while higher polynomial degree increases nonlinearity. section 4.4.3

Figure 4.17 Impact of model complexity on the fitted model’s training and testing performance.

Compare how low- and high-complexity polynomial models affect fitted curves, training performance, and testing performance. section 4.4.1

Figure 4.20 Validation curves for SVR and preprocessing hyperparameters.

Inspect how SVR regularization, polynomial degree, and error-margin settings alter complexity and generalization. section 4.4.3

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains complexity through parameters, features, polynomial degree, regularization, bias, variance, and validation curves.
Examples
Uses polynomial fits, SVR hyperparameters, hERG classification, and AqSolDB solubility prediction to illustrate complexity.
What it assumes
Assumes basic supervised-learning and prediction concepts while defining capacity, bias, variance, underfitting, and overfitting.
What you can do after
Evaluate complexity choices with training and test errors, validation curves, bias, variance, and generalization behavior.
How it connects
Connects model complexity to capacity, bias-variance trade-offs, underfitting, overfitting, regularization, validation curves, and applicability domains.

Also explained in

Compare how 4 books explain model complexity →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.