liveBook · Manning

Early stopping

As explained in Timeless Algorithms, The Foundational Ideas

Early stopping is a regularization technique that halts training when further learning is likely to harm performance on held-out data.

Early stopping regularizes a network by ending training when validation performance stops improving, even if training loss continues to decline. It limits further adjustment to training-specific details. section 11.6.2

Like pruning, early stopping restricts excessive adaptation; unlike pruning, it limits continued parameter adjustment rather than removing branches. Validation data guide stopping decisions, while test data provide a final evaluation after those decisions. section 11.6.2

Stopping limits one route to overfitting, but does not establish that performance will carry to production conditions absent from the validation data. section 11.6.2

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Defines a validation-based stopping condition and its overfitting rationale, without specifying an implementation rule.
Examples
Contrasts improving validation performance with declining training loss as the signal to stop.
What you can do after
Understand when continued training may harm generalization and why validation performance, not training loss alone, guides stopping.
How it connects
Connects regularization to overfitting, validation data, training loss, pruning, dropout, weight decay, data augmentation, support vector machines, and random forests.

Also explained in

Compare how 11 books explain early stopping →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.