liveBook · Manning

Overfitting

As explained in Build AI Drug Discovery Pipelines

Overfitting is the condition in which a model fits training examples too closely and performs poorly on new data.

Overfitting occurs when a model memorizes training examples instead of learning patterns that work on new data. Its performance on unseen inputs suffers, even if it fits the training set well. The ability to perform well on new data is called generalization. section 1.2.2

Recognizing dogs and molecules

A learner shown only German Shepherds might fail to recognize a poodle as a dog. A molecular model faces a similar risk: labeled compounds are expensive to assemble, chemical space is vast, and drug discovery often explores structurally diverse compounds. section 1.2.2

Monitor overfitting by comparing training loss with loss on a held-out validation set after each training epoch. A persistent gap, or validation loss that stops improving while training loss continues to fall, signals a potential generalization problem. A separate test set assesses performance on new data. section 8.2.3 section 1.2.2

Figure 8.13 Logging the train and validation loss with each epoch of training. The validation loss saturates before 50 epochs, which informs further hyperparameter tweaking (e.g., cut down the number of epochs or use early stopping, adjust the learning rate or scheduler, etc.).

The loss curves show validation loss saturating before 50 epochs while training continues, suggesting that training duration or other hyperparameters should be reconsidered. section 8.2.3

Possible responses depend on the model and the observed curves:

  • For a linear model, use regularization to trade some training-set accuracy for smaller coefficient weights. section 3.5
  • For a neural network whose validation loss has saturated, consider fewer epochs, early stopping, or changes to the learning rate or scheduler. section 8.2.3
  • Use learning and validation curves to examine errors alongside the bias-variance trade-off. section 4.4.3

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Defines overfitting and demonstrates loss monitoring, with mitigation through regularization and training adjustments.
Examples
Uses dog-breed recognition and neural-network training and validation losses to illustrate generalization problems.
What you can do after
Diagnose potential overfitting from training and validation losses and consider regularization or training adjustments.
How it connects
Connects generalization, regularization, validation curves, and the bias-variance trade-off.

Also explained in

Compare how 33 books explain overfitting →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.