Underfitting

As explained in Machine Learning System Design

A condition in which a model is too simple to capture important relationships between features and the target variable, resulting in poor performance on both training and new data.

Underfitting occurs when a model is too simple to capture important relationships between features and the target, producing poor results on both training and new data. It reflects high bias: simplified assumptions prevent the model from capturing useful signals. Unlike overfitting, it is not mainly a failure to generalize from training data. section 9.1.1

Use learning curves to distinguish underfitting from data and variance problems:

• When training and validation curves nearly converge while both losses remain high, treat the pattern as high bias and underfitting. section 9.1.5
• Do not expect more training data to fix this pattern; test a more complicated model instead. section 9.1.5
• Interpret a large training–validation gap as high variance or an unfair data split rather than underfitting. section 9.1.5
• Check convergence first; if the model has not converged or shows underfitting, continue debugging before deeper error analysis. section 9.1 section 9.5

When an underfitted model is harmless

An early foot-detection model in a virtual try-on application sometimes placed shoes on pet paws and toys. Users initially found the mistakes funny, but the model was later improved for conventional scenarios. section 2.4

Figure 9.2 With an increasing number of model parameters, training error tends to become lower and lower while minimizing bias. At the same time, the model variance increases, providing us with a U-shaped validation error.

Shows how increasing model parameters can lower training error while increasing variance and producing a U-shaped validation error. section 9.1

How Machine Learning System Design covers it

How deep it goes

Defines high bias and contrasts it with overfitting, then provides curve-based remedies without deriving model internals.

Examples

Uses training–validation curves and an augmented-reality footwear detector that mistakes pet paws and toys for feet.

What it assumes

Assumes training and validation metrics can be compared; teaches bias, variance, convergence, and curve-based diagnosis.

What you can do after

Diagnose underfitting from converged, high-loss curves and choose greater model complexity instead of collecting more data.

How it connects

Connects underfitting with overfitting, the bias–variance tradeoff, learning curve analysis, error analysis, feature engineering, and hyperparameter tuning.

liveBookmanning.com

Read Machine Learning System Design on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Grokking Machine LearningA model whose complexity is insufficient to capture the relevant structure of its dataset.
• 

  liveBookmanning.com
  Generative AI in ActionUnderfitting is a modeling condition in which a model fails to learn enough structure from its training data to perform well on new examples.
• 

  liveBookmanning.com
  Grokking AI Algorithms, Second EditionUnderfitting is a condition in which a model is too simple or insufficiently trained to perform well even on its training data.
• 

  liveBookmanning.com
  Deep Learning with R, Third EditionUnderfitting is a modeling condition in which a model is too simple or insufficiently trained to capture patterns needed for good performance.
• 

  liveBookmanning.com
  Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIUnderfitting is a modeling failure in which a model is too simple to capture important patterns in the data.
• 

  liveBookmanning.com
  The AI Pocket BookA failure mode where a model is too simple to capture the training pattern and performs poorly.

Compare how 7 books explain underfitting →

Related concepts

• overfittingcontrasts with · Overfitting is a modeling condition in which a model fits patterns or idiosyncrasies in its training data too closely and consequently generalizes poorly to new data.
• genetic algorithmaddressed by · A genetic algorithm is an evolutionary optimization method that improves a population of candidate solutions through fitness-based selection and variation, often including recombination and mutation.
• bias-variance tradeoffcontrasts with · The bias-variance tradeoff is the balance between systematic error from restrictive model assumptions and error from sensitivity to variation in sampled data.
• learning curvemeasured by · A learning curve is a representation of how performance or proficiency changes as experience, training progress, or training data increases.
• expressive poweraddresses · Expressive power is the capacity of a model, language, or system to represent or convey a range and complexity of functions, patterns, distinctions, or ideas.

This explainer was generated by Manning's knowledge graph from the text of Machine Learning System Design. The section links point to the parts of the book it draws on.