Underfitting

As explained in Deep Learning with R, Third Edition

Underfitting is a modeling condition in which a model is too simple or insufficiently trained to capture patterns needed for good performance.

Underfitting occurs while training still has meaningful progress to make: the model has not yet captured all relevant patterns, so training and validation performance can improve together. section 5.1.1

Recognize underfitting through these training patterns:

• Training loss does not decrease, indicating that training has not properly started. section 5.3
• Training progresses but fails to beat a commonsense or trivial baseline, showing weak generalization. section 5.3
• Training and validation loss both decrease, yet the model cannot overfit; this indicates that the model remains underfit. section 5.3

Finding sufficient model capacity

A logistic regression model can have statistical power on MNIST without solving the problem well. Increase capacity by adding layers or making layers larger, then train for more epochs. Monitor training and validation losses and metrics until validation performance begins to degrade; crossing that point confirms that the model can overfit and helps locate the boundary between underfitting and overfitting. section 6.2.4

Figure 5.1 Canonical overfitting behavior

Use the canonical training-versus-validation curve to locate the transition from improving underfit behavior to degrading overfit behavior. section 5.1.1

How Deep Learning with R, Third Edition covers it

How deep it goes

Explains training-loss and validation-loss patterns, then gives capacity-scaling steps without mathematical derivation.

Examples

Uses MNIST, logistic regression, training curves, trivial baselines, and progressively larger neural networks as examples.

What it assumes

Assumes familiarity with training, validation data, loss, metrics, baselines, and overfitting while introducing the diagnosis through examples.

What you can do after

Diagnose underfitting from loss and metric trends, then increase layers, layer size, or training duration to test model capacity.

How it connects

Connects underfitting with optimization, generalization, overfitting, model capacity, validation loss, and baseline performance.

liveBookmanning.com

Read Deep Learning with R, Third Edition on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Grokking Machine LearningA model whose complexity is insufficient to capture the relevant structure of its dataset.
• 

  liveBookmanning.com
  Machine Learning System DesignA condition in which a model is too simple to capture important relationships between features and the target variable, resulting in poor performance on both training and new data.
• 

  liveBookmanning.com
  Generative AI in ActionUnderfitting is a modeling condition in which a model fails to learn enough structure from its training data to perform well on new examples.
• 

  liveBookmanning.com
  Grokking AI Algorithms, Second EditionUnderfitting is a condition in which a model is too simple or insufficiently trained to perform well even on its training data.
• 

  liveBookmanning.com
  Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIUnderfitting is a modeling failure in which a model is too simple to capture important patterns in the data.
• 

  liveBookmanning.com
  The AI Pocket BookA failure mode where a model is too simple to capture the training pattern and performs poorly.

Compare how 7 books explain underfitting →

Related concepts

• overfittingcontrasts with · Overfitting is a modeling condition in which a model fits patterns or idiosyncrasies in its training data too closely and consequently generalizes poorly to new data.
• genetic algorithmaddressed by · A genetic algorithm is an evolutionary optimization method that improves a population of candidate solutions through fitness-based selection and variation, often including recombination and mutation.
• bias-variance tradeoffcontrasts with · The bias-variance tradeoff is the balance between systematic error from restrictive model assumptions and error from sensitivity to variation in sampled data.
• learning curvemeasured by · A learning curve is a representation of how performance or proficiency changes as experience, training progress, or training data increases.
• expressive poweraddresses · Expressive power is the capacity of a model, language, or system to represent or convey a range and complexity of functions, patterns, distinctions, or ideas.

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with R, Third Edition. The section links point to the parts of the book it draws on.