Underfitting

As explained in Grokking AI Algorithms, Second Edition

Underfitting is a condition in which a model is too simple or insufficiently trained to perform well even on its training data.

Underfitting happens when a model is too simple or insufficiently trained to learn useful patterns. It performs poorly on both its training data and new data. section 8.13 section 8.17

A useful contrast is overfitting. An overfit model memorizes training examples and struggles with variation in new data; an underfit model fails to learn enough from either set. A good fit captures concepts rather than memorizing details. section 8.13

Use performance patterns to distinguish the three cases:

• Underfitting produces many incorrect classifications on training and unseen data. section 8.17
• Overfitting produces very high training accuracy but poor performance on new, unseen data. section 8.13 section 8.17
• An ideal fit lies between these extremes and generalizes beyond the examples used for training. section 8.13 section 8.17

Studying for an exam

Imagine three students preparing for an exam. One does not study and fails both the practice test and the real exam: this is underfitting. Another memorizes the practice-test answers, scores perfectly there, and fails when questions are worded differently: this is overfitting. A student who understands the concepts may miss details but can solve unfamiliar questions: this is a good fit. section 8.13

Figure 8.26 Underfitting, ideal fit, and overfitting

Compare underfitting, ideal fit, and overfitting visually to connect error patterns with model generalization. section 8.17

How Grokking AI Algorithms, Second Edition covers it

How deep it goes

Defines underfitting through contrasts and an analogy, then stops short of prescribing a specific remedy.

Examples

Uses an exam-study analogy, incorrect classifications, training accuracy, unseen-data performance, and a fit-comparison figure.

What it assumes

Assumes basic familiarity with models, training data, and accuracy while explaining fit through everyday examples.

What you can do after

Evaluate model fit by comparing training performance with performance on new, unseen data.

How it connects

Connects underfitting with overfitting, generalization, training data, unseen data, decision trees, and pruning.

liveBookmanning.com

Read Grokking AI Algorithms, Second Edition on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Grokking Machine LearningA model whose complexity is insufficient to capture the relevant structure of its dataset.
• 

  liveBookmanning.com
  Machine Learning System DesignA condition in which a model is too simple to capture important relationships between features and the target variable, resulting in poor performance on both training and new data.
• 

  liveBookmanning.com
  Generative AI in ActionUnderfitting is a modeling condition in which a model fails to learn enough structure from its training data to perform well on new examples.
• 

  liveBookmanning.com
  Deep Learning with R, Third EditionUnderfitting is a modeling condition in which a model is too simple or insufficiently trained to capture patterns needed for good performance.
• 

  liveBookmanning.com
  Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIUnderfitting is a modeling failure in which a model is too simple to capture important patterns in the data.
• 

  liveBookmanning.com
  The AI Pocket BookA failure mode where a model is too simple to capture the training pattern and performs poorly.

Compare how 7 books explain underfitting →

Related concepts

• overfittingcontrasts with · Overfitting is a modeling condition in which a model fits patterns or idiosyncrasies in its training data too closely and consequently generalizes poorly to new data.
• genetic algorithmaddressed by · A genetic algorithm is an evolutionary optimization method that improves a population of candidate solutions through fitness-based selection and variation, often including recombination and mutation.
• bias-variance tradeoffcontrasts with · The bias-variance tradeoff is the balance between systematic error from restrictive model assumptions and error from sensitivity to variation in sampled data.
• learning curvemeasured by · A learning curve is a representation of how performance or proficiency changes as experience, training progress, or training data increases.
• expressive poweraddresses · Expressive power is the capacity of a model, language, or system to represent or convey a range and complexity of functions, patterns, distinctions, or ideas.

This explainer was generated by Manning's knowledge graph from the text of Grokking AI Algorithms, Second Edition. The section links point to the parts of the book it draws on.