liveBook · Manning

Learning curve

As explained in Build AI Drug Discovery Pipelines

A learning curve is a diagnostic plot showing model performance or error as a function of training-set size.

A learning curve plots training and test error against training-set size. It reveals whether a model’s performance is limited mainly by bias, variance, or insufficient data. section 4.4.2 section 4.4.1

Read the curves by comparing their levels and the gap between them:

  • Few training samples often produce low training error but high test error, indicating overfitting and high variance. section 4.4.2
  • As the dataset grows, training error commonly rises while test error falls; convergence suggests that additional data may provide little benefit. section 4.4.2
  • A higher converged error indicates greater bias, while a persistent train-test gap indicates variance that may require more data or regularization. section 4.4.2
  • Construct curves by measuring a chosen score as training size changes, commonly with Scikit-Learn cross-validation. section 4.4.1

Solubility models

For solubility prediction, RANSAC showed higher training MSE, consistent with higher bias, but its train-test gap shrank toward zero. SVR had lower bias and higher variance; additional training data could therefore reduce its overfitting. section 4.4.2

Figure 4.19 Learning curves for RANSAC and SVR models.

Inspect the plotted learning curves for the RANSAC and SVR solubility models. section 4.4.2

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains train-test behavior across dataset sizes and stops short of prescribing a single model-selection rule.
Examples
Uses solubility-prediction learning curves comparing RANSAC and SVR with training and test mean squared error.
What it assumes
Assumes basic training and test errors, while explaining bias, variance, overfitting, and convergence through model behavior.
What you can do after
Diagnose whether more data, a more complex model, or regularization is the most appropriate next intervention.
How it connects
Connects overfitting, bias, variance, model complexity, regularization, validation curves, and cross-validation.

Also explained in

Compare how 3 books explain learning curve →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.