Overfitting
Overfitting is a modeling condition in which a model adapts too closely to particular data, including its noise or idiosyncrasies, and consequently generalizes poorly to new data.
Grokking Machine Learning
Overfitting is a modeling condition in which a model captures noise or idiosyncrasies in training data and generalizes poorly to new data.
- How it is used
- Diagnoses models that fit training data well but make poor predictions on held-out data.
- How deep it goes
- Works through polynomial fits, error comparisons, data splits, complexity graphs, regularization, and brief model-specific prevention techniques.
- Examples
- Contrasts degree-1, degree-2, and degree-10 curves, then asks readers to classify models from training and testing errors.
- What you can do after
- Diagnose/debug with training and held-out errors, then use validation and complexity controls to address overfitting.
Deep Learning with Python, Third Edition
The failure mode where performance improves on training data while degrading on unseen data because the model has adapted too specifically to the training set.
- How it is used
- Uses overfitting as a signal for locating model capacity and shifting from optimization to generalization.
- How deep it goes
- Explains divergent training and validation curves, validation-set leakage, and practical mitigation across several model types.
- Examples
- Uses sentiment classification curves, a small image dataset, and LSTM temperature forecasting to illustrate overfitting.
- What you can do after
- Diagnose overfitting with training and validation curves, then select mitigation and preserve a separate test set for evaluation.
Read how Deep Learning with Python, Third Edition explains overfitting →
Deep Learning with R, Third Edition
Overfitting is a modeling condition in which performance is better on training data than on previously unseen data, including when repeated tuning adapts a model to validation data.
- How it is used
- Uses overfitting to locate the boundary between fitting a model and improving its performance on unseen data.
- How deep it goes
- Explains training and validation divergence, possible sources of misleading patterns, and a workflow for detecting and reducing overfitting.
- Examples
- Illustrates overfitting with MNIST digit classification, IMDb training curves, mislabeled digits, and ambiguous banana ripeness labels.
- What you can do after
- Diagnoses overfitting using training and validation metrics, then adjusts training duration, model size, or regularization.
Read how Deep Learning with R, Third Edition explains overfitting →
Deep Learning with PyTorch, Second Edition
A failure mode where a model memorizes irrelevant details of the training data and generalizes poorly, discussed here as a risk of increased capacity.
- How it is used
- Uses overfitting to guide model capacity choices and assess whether training improvements generalize to unseen samples.
- How deep it goes
- Explains memorization and capacity, compares training and validation loss curves, and discusses data augmentation and regularization as responses.
- Examples
- Illustrates memorized faces in age prediction and class-specific loss divergence in a lung nodule classifier.
- What you can do after
- Diagnose/debug with training and validation loss comparisons, including class-specific losses when overall metrics conceal divergence.
Read how Deep Learning with PyTorch, Second Edition explains overfitting →
Build AI Drug Discovery Pipelines
Overfitting is the condition in which a model fits training examples too closely and performs poorly on new data.
- How it is used
- Frames generalization as essential when predicting properties of structurally diverse compounds in drug discovery.
- How deep it goes
- Defines overfitting and demonstrates loss monitoring, with mitigation through regularization and training adjustments.
- Examples
- Uses dog-breed recognition and neural-network training and validation losses to illustrate generalization problems.
- What you can do after
- Diagnose potential overfitting from training and validation losses and consider regularization or training adjustments.
Read how Build AI Drug Discovery Pipelines explains overfitting →
Grokking Machine Learning, Second Edition
Overfitting is a modeling failure in which a learned model fits training data too closely and performs poorly on new data.
- How it is used
- Uses overfitting to motivate testing, validation, complexity selection, and regularization.
- How deep it goes
- Explains overfitting with polynomial, recommendation, and decision-tree examples, then connects it to validation errors and regularization.
- Examples
- Illustrates it with noisy parabola data, movie recommendations, and a student-admissions decision tree.
- What you can do after
- Diagnose/debug with it by recognizing strong training performance paired with poor validation performance and selecting less excessive complexity.
Read how Grokking Machine Learning, Second Edition explains overfitting →
Causal Inference for Data Science
Overfitting is the condition in which a model captures noise or idiosyncrasies in training data and consequently generalizes poorly to new data.
- How it is used
- Diagnoses why strong historical fit can fail for future prediction and why causal analyses require additional protection against same-data prediction.
- How deep it goes
- Explains flexibility, noise, conditional expectation, train-test splitting, cross-validation, and cross-fitting, while stopping short of model-specific algorithms.
- Examples
- Uses a solved-exam analogy, train-test prediction, varying-complexity fits, license-plate recognition, and two-fold causal learners.
- What you can do after
- Diagnose overfitting with held-out evaluation and apply cross-fitting when causal predictions target observations used in analysis.
Read how Causal Inference for Data Science explains overfitting →
Natural Language Processing in Action, Second Edition
Overfitting is a modeling failure in which a model relies too heavily on peculiarities of its training data and performs poorly on new data.
- How it is used
- Uses overfitting to diagnose why NLP classifiers perform well on training examples but fail on varied, unseen language.
- How deep it goes
- Explains causes, accuracy symptoms, parameter effects, and countermeasures, stopping short of a formal statistical treatment.
- Examples
- Uses toxicity filtering, disaster-post CNNs, convolution-channel comparisons, and documented RNN hyperparameter experiments.
- What you can do after
- Diagnose/debug with it by comparing training and test results, inspecting parameters, and adjusting capacity, representations, dropout, or training settings.
Read how Natural Language Processing in Action, Second Edition explains overfitting →
Math and Architectures of Deep Learning
A training failure mode in which a model fits idiosyncrasies of the training data too closely and generalizes poorly.
- How it is used
- Explains why neural networks that fit training data too closely fail to generalize during testing or inference.
- How deep it goes
- Connects capacity, decision boundaries, symptoms, regularization, and dropout, but stops short of a full optimization procedure.
- Examples
- Uses anomalous binary-classification points, simpler versus wriggly boundaries, and a PyTorch dropout model.
- What you can do after
- Diagnose overfitting from training-test performance and apply regularization or dropout to reduce memorization.
Read how Math and Architectures of Deep Learning explains overfitting →
Grokking AI Algorithms, Second Edition
Overfitting is a condition in which a model matches training data too closely and performs poorly on varied unseen data.
- How it is used
- Uses overfitting to explain why strong training performance may not generalize to unseen data.
- How deep it goes
- Defines overfitting, contrasts underfitting, and explains pruning and validation-based early stopping without implementing those remedies.
- Examples
- Uses a practice-test analogy, decision-tree branches, and an early-stopping patience of three evaluations.
- What you can do after
- Diagnose overfitting by comparing training performance with unseen-data performance or validation loss.
Read how Grokking AI Algorithms, Second Edition explains overfitting →
Building Specialized AI Systems
Overfitting is a modeling failure in which a model memorizes training examples instead of learning patterns that generalize.
- How it is used
- Guards task-specific fine-tuning against memorizing examples instead of adapting to a target domain.
- How deep it goes
- Defines memorization versus generalization and gives epoch control, early stopping, logging, and evaluation as practical safeguards.
- Examples
- Uses analyst report-to-summary prompt-target pairs and checks for overfitting to specific phrasing or patterns.
- What you can do after
- Evaluate/understand whether a fine-tuned model is learning general task patterns rather than memorizing its training examples.
Read how Building Specialized AI Systems explains overfitting →
Ensemble Methods for Machine Learning
Overfitting is a modeling failure in which a model fits training data too closely and does not necessarily generalize well to new data.
- How it is used
- Diagnoses the gap between fitting training data and generalizing to future data during model selection and deployment.
- How deep it goes
- Defines the train-test failure, contrasts underfitting, and connects it to the bias-variance tradeoff without formal proofs.
- Examples
- Uses synthetic regression, RBF-kernel SVMs, bagged decision trees, LightGBM, CatBoost, and XGBoost training curves.
- What you can do after
- Diagnose overfitting with train-test comparisons and control it using cross-validation, aggregation, learning rates, or early stopping.
Read how Ensemble Methods for Machine Learning explains overfitting →
The AI Pocket Book
A failure mode in which a model becomes too specialized to its training data and generalizes poorly to new cases.
- How it is used
- Frames the gap between low training loss and reliable performance on unseen data.
- How deep it goes
- Defines memorization, explains its generalization failure, and lists early stopping, limited updates, regularization, and dropout.
- Examples
- Uses fine-tuned language models that memorize style examples and contrasts training performance with unseen sentences or images.
- What you can do after
- Diagnose overfitting by comparing training, validation, and test performance, then apply named fine-tuning controls.
Learn Generative AI with PyTorch
A training failure mode where the model memorizes training data too closely and generalizes poorly to new inputs.
- How it is used
- Uses validation performance and checkpoint comparison to choose training duration without sacrificing performance on unseen data.
- How deep it goes
- Defines the failure mode, demonstrates its effects, and gives validation-loss early stopping, but does not develop broader regularization methods.
- Examples
- Demonstrates image classification with train-validation-test splits and GPT text generation that copies training passages.
- What you can do after
- Diagnose overfitting with validation loss and control it by selecting checkpoints or stopping training after a patience interval.
Read how Learn Generative AI with PyTorch explains overfitting →
Vibe Engineering
Overfitting is a modeling failure in which learned patterns fit training data too closely and generalize poorly to unseen data.
- How it is used
- Uses overfitting as a warning when a medical risk classifier produces exceptionally strong precision and recall curves.
- How deep it goes
- Explains memorization, proposes five-fold cross-validation, and describes dropout as a preventive measure.
- Examples
- Uses high-risk medicine descriptions, nearly flat metric curves, and five fold-accuracy results to illustrate the concern.
- What you can do after
- Evaluate whether strong classifier scores warrant an overfitting check and interpret them alongside threshold tradeoffs.
Data Without Labels
Overfitting is a modeling condition in which a supervised learning model performs well on training data but less accurately on unseen data.
- How it is used
- Uses the training–unseen-data gap to explain why models fail to generalize.
- How deep it goes
- Defines the condition, links it to dimensionality, and gives PCA/SVD as mitigations without detailing model diagnostics.
- Examples
- Uses insurance customer variables and the Hughes performance curve to illustrate irrelevant dimensions and declining model performance.
- What you can do after
- Apply PCA or SVD to reduce variables and mitigate overfitting, while recognizing possible information loss and reduced interpretability.
Investing for Programmers
Overfitting is a modeling failure in which a model captures irrelevant variation in training features and fails to generalize reliably to new data.
- How it is used
- Uses held-out testing and feature selection to frame generalization risk in financial supervised learning.
- How deep it goes
- Defines overfitting as learning noise, contrasts underfitting, and stops short of detailed mitigation techniques.
- Examples
- Demonstrates next-day stock-price prediction with historical prices, a random forest, RMSE, and an actual-versus-forecast plot.
- What you can do after
- Diagnose/debug with it by comparing held-out predictions with RMSE and questioning whether features represent signal or noise.
Generative AI in Action
Overfitting is a modeling condition in which learned patterns fit training data too closely and fail to generalize well to new examples.
- How it is used
- Identifies generalization risks in LLM fine-tuning and an analogous overspecification risk in prompt engineering.
- How deep it goes
- Explains validation and hyperparameter adjustment, while deferring detailed benchmarks and evaluation tools.
- Examples
- Uses an emoji chatbot JSONL dataset to illustrate examples for fine-tuning.
- What you can do after
- Evaluate generalization by monitoring validation performance, adjusting hyperparameters, and comparing fine-tuned models against baselines.
LLM Customization and Fine-Tuning
A model-learning failure in which performance worsens on inputs that differ from the training distribution because the model has fit the training data too narrowly.
- How it is used
- Frames overfitting as a production generalization risk in fine-tuning and prompt selection.
- How deep it goes
- Explains loss-curve diagnosis and gives early-stopping, regularization, data-diversity, and capacity-reduction remedies with practical starting values.
- Examples
- Uses a 0.3 training versus 0.9 validation loss example and a prompt scoring 89% on its design examples.
- What you can do after
- Diagnoses overfitting with loss curves and mitigates it through early stopping, regularization, more diverse data, or reduced model capacity.
Read how LLM Customization and Fine-Tuning explains overfitting →
Machine Learning System Design
A condition in which a model fits its training data too closely and performs poorly on unseen data because it captures insignificant or non-generalizable patterns.
- How it is used
- Diagnoses poor generalization by comparing training performance with unseen-data performance during error analysis.
- How deep it goes
- Defines the condition, explains bias–variance causes, identifies contributing data problems, and prescribes capacity or regularization changes.
- Examples
- Uses training-versus-validation loss curves and increasing model complexity to illustrate overfitting and rising variance.
- What you can do after
- Diagnose overfitting from learning or loss curves and respond by reducing model capacity or increasing regularization.
Read how Machine Learning System Design explains overfitting →
Timeless Algorithms, The Foundational Ideas
Overfitting occurs when a model fits its training observations too closely without capturing relationships that remain useful beyond them.
- How it is used
- Frames the gap between training fit and performance on new observations as a central test of learning.
- How deep it goes
- Defines overfitting and explains its causes through tree depth, sample variance, model capacity, and regularization.
- Examples
- Uses deep tree leaves, neural-network capacity, SVM penalty selection, and language-model performance on held-out text.
- What you can do after
- Diagnose overfitting by comparing validation performance and preserve an untouched test set for final evaluation.
Read how Timeless Algorithms, The Foundational Ideas explains overfitting →
Starting Data Analytics with Generative AI and Python
Overfitting is a modeling failure in which a model fits noise or sample-specific variation rather than generalizable underlying patterns.
- How it is used
- Frames model complexity as a practical reliability risk in regression analysis of customer review scores.
- How deep it goes
- Defines the failure and its parameter-to-observation cause, but stops short of detailed mitigation procedures.
- Examples
- Uses a customer review-score model involving delivery, product, price, and payment attributes.
- What you can do after
- Evaluate model reliability by limiting unnecessary predictors and checking held-out predictions, accuracy, and confusion matrices.
Read how Starting Data Analytics with Generative AI and Python explains overfitting →
Local Image Generation
Overfitting is a learning failure in which a model memorizes training examples or patterns too specifically and generalizes poorly to new inputs.
- How it is used
- Uses sample prompts to judge whether a LoRA training job memorizes its dataset.
- How deep it goes
- Explains practical detection through prompt selection and checkpoint comparison, without detailing model internals.
- Examples
- Uses familiar Keats-the-dog scenes and novel-environment prompts to expose memorization.
- What you can do after
- Diagnose/debug with sample prompts and select the checkpoint that generalizes best.
Transfer Learning for Natural Language Processing
Overfitting occurs when a model fits its training data too closely and consequently generalizes poorly to new data.
- How it is used
- Monitors generalization risk when evaluating gradient-boosted classifiers and deciding how much of a pretrained network to fine-tune.
- How deep it goes
- Defines overfitting, describes a training-versus-validation warning sign, and outlines data, model-complexity, and fine-tuning responses.
- Examples
- Uses gradient-boosting training, cross-validation, and test scores, plus layer-freezing decisions for fine-tuning on limited target data.
- What you can do after
- Evaluate generalization risk by checking cross-validation and test results, and limit fine-tuning parameters to what the data can support.
Read how Transfer Learning for Natural Language Processing explains overfitting →
Demand Forecasting Best Practices
Overfitting is a modeling condition in which a model fits historical data well but fails to generalize to future or otherwise unseen data.
- How it is used
- Protects demand-planning decisions from forecasts that appear accurate historically but fail when applied to future demand.
- How deep it goes
- Explains causes, benchmark comparisons, data-history risks, and visual contrast with underfitting, but stops short of implementation details.
- Examples
- Uses a 15-model selection example, three-year historical demand, nonexistent seasonality, and benchmark comparisons across 2022 and 2023.
- What you can do after
- Diagnose overfitting by separating optimization from evaluation, matching forecast lags, comparing benchmarks, and monitoring future performance.
Read how Demand Forecasting Best Practices explains overfitting →
Sutskever's List
Overfitting is a modeling failure in which a model fits training data too closely and performs poorly on unseen data.
- How it is used
- Explains why neural networks struggled with limited data and how training choices can preserve generalization.
- How deep it goes
- Defines overfitting as fitting noise, then connects it to model complexity and generalization without offering quantitative diagnostics.
- Examples
- Uses AlexNet’s 60-million-parameter, 1.2-million-image training setup and a lookup-table-versus-straight-line contrast.
- What you can do after
- Understand how augmentation and dropout can reduce overfitting, and how excessive model complexity can memorize noise.
Statistics Every Programmer Needs
Overfitting is a modeling condition in which a model captures idiosyncrasies of the fitting data at the expense of performance on new data.
- How it is used
- Frames regression evaluation as a generalization problem rather than accepting a higher R² as automatic evidence of better prediction.
- How deep it goes
- Defines noise-fitting and explains predictor-count and outlier-removal risks, but stops short of presenting a complete prevention workflow.
- Examples
- Uses added predictors and removal of one outlier that raises R² from 0.88 to 0.95 as concrete overfitting examples.
- What you can do after
- Evaluate regression fit more cautiously by distinguishing higher R² from reliable performance on new data.
Read how Statistics Every Programmer Needs explains overfitting →
Fight Fraud with Machine Learning
Overfitting is a modeling condition in which a model captures training-specific patterns that reduce performance on unseen data.
- How it is used
- Diagnoses why receipt-forgery predictions perform worse on unseen images than on training images.
- How deep it goes
- Explains overfitting through train/test heatmaps and demonstrates multi-task mitigation, without developing a general theoretical treatment.
- Examples
- Uses receipt forgery bounding boxes, probability heatmaps, item-price regions, textual notes, and auxiliary text-detection predictions.
- What you can do after
- Diagnose overfitting by comparing train/test heatmaps and apply multi-task learning or pipeline changes to improve generalization.
Read how Fight Fraud with Machine Learning explains overfitting →
LLMs in Production
Overfitting is a condition in which a model becomes excessively tailored to training data and performs poorly on unseen data.
- How it is used
- Uses training monitoring to preserve language-model performance across unseen data and broad contexts.
- How deep it goes
- Defines the failure mode and names three safeguards, but stops short of implementation details or diagnostic thresholds.
- Examples
- Uses language-model training, unseen-data performance, and monitoring alongside early stopping, dropout, and learning rate scheduling.
- What you can do after
- Diagnose/debug with it by monitoring training and applying early stopping, dropout, or learning rate scheduling.
Graph Neural Networks in Action
A modeling failure mode in which a model fits training data patterns too closely and generalizes poorly to validation or unseen data.
- How it is used
- Diagnoses graph-model generalization during training by comparing training and validation performance.
- How deep it goes
- Explains curve divergence, validation-loss minima, and checkpoint selection, but defers corrective techniques to later treatment.
- Examples
- Uses GCN and GraphSAGE loss curves, epoch-20 divergence, validation checkpoints, and product-category classification metrics.
- What you can do after
- Diagnose overfitting with loss curves, select the lowest-validation-loss checkpoint, and assess generalization on test data.
Read how Graph Neural Networks in Action explains overfitting →
Reinforcement Learning from Human Feedback
Overfitting is a training condition in which a model fits its training data or objective too closely and generalizes poorly.
- How it is used
- Controls prompt-level data imbalance when training reward models from multiple completion comparisons.
- How deep it goes
- Defines memorization-based generalization failure and gives a batching and loss-weighting remedy, but does not develop model-level regularization.
- Examples
- Uses correlated prompt comparisons, verbose responses, and repeated rare tokens as concrete failure cases.
- What you can do after
- Diagnose prompt-level overfitting and mitigate it with grouped batches and normalized comparison losses.
Read how Reinforcement Learning from Human Feedback explains overfitting →
Building Reliable AI Systems
A model-training outcome in which a model fits training examples too closely and generalizes poorly to new data.
- How it is used
- Reduces narrow learning during fine-tuning by guiding dataset preparation toward the range of inputs the model will encounter.
- How deep it goes
- Recommends broad data coverage and balanced representation, without detailing model-level mechanisms.
- Examples
- Uses varied support issues, repeated customer questions, diverse conversation phrasings, and topics for style training.
- What you can do after
- Apply dataset-curation practices that broaden input coverage and limit overrepresentation of narrow subcategories.
Read how Building Reliable AI Systems explains overfitting →
Financial AI in Practice
Overfitting occurs when a model captures patterns specific to its training data that do not generalize reliably to new data.
- How it is used
- Uses overfitting as a reason to scrutinize market-prediction accuracy and constrain model complexity.
- How deep it goes
- Explains historical memorization and random-forest safeguards, then evaluates out-of-sample predictions without covering full trading economics.
- Examples
- Contrasts memorized historical market sequences with a conservatively configured random forest reporting 52.2% directional accuracy.
- What you can do after
- Apply it via scikit-learn by limiting random-forest depth, sample thresholds, and feature sampling, then evaluating out of sample.
Also covered in
- Data-Driven Cybersecurity
- Machines that ThinkIn early access (MEAP)
- Build a Text-to-Image Generator (from Scratch)
- Build an Advanced RAG Application (From Scratch)In early access (MEAP)
- Statistics Slam Dunk
- AI-Powered Search
- Rearchitecting LLMsIn early access (MEAP)
- Experimentation for Engineers
- Grokking Statistics
- Grokking BayesIn early access (MEAP)
- AI Model EvaluationIn early access (MEAP)
- AI Recommender SystemsIn early access (MEAP)
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- objective functionAn objective function is a mathematical function that assigns values to candidate solutions or parameter settings, which an optimization procedure seeks to minimize or maximize.
- generationGeneration is the production of an output by a computational model from supplied input or context, often using patterns learned from data.
- accuracyAccuracy is the degree to which a result agrees with a specified truth, reference, or criterion, commonly measured as the proportion of evaluated predictions that are correct.
Related concepts
- dropoutaddressed by · Dropout is a neural-network regularization technique that randomly suppresses selected units or activations during training to reduce overfitting.
- regularizationaddressed by · Regularization is a modeling technique that constrains learning or penalizes model complexity to discourage overfitting and improve generalization.
- early stoppingaddressed by · Early stopping is a technique that terminates an iterative process before its planned completion when a specified condition indicates that further work is unnecessary or unlikely to improve the objective.
- underfittingcontrasts with · A modeling failure in which insufficient capacity or training prevents a model from capturing relevant patterns in the data, resulting in poor performance even on training data.
- failure modekind of · A failure mode is a distinct way in which a system, component, process, or operation can fail or produce an undesirable result.
- tree pruningaddressed by · Tree pruning is the removal of decision-tree branches that fail a specified split-quality condition.
Questions about overfitting
What is overfitting?
Overfitting is a modeling condition in which a model adapts too closely to particular data, including its noise or idiosyncrasies, and consequently generalizes poorly to new data.
Which Manning books explain overfitting?
Grokking Machine Learning, Deep Learning with Python, Third Edition, Deep Learning with R, Third Edition, Deep Learning with PyTorch, Second Edition, Build AI Drug Discovery Pipelines, Grokking Machine Learning, Second Edition, Causal Inference for Data Science, Natural Language Processing in Action, Second Edition and 25 more.
What should I know before learning overfitting?
Manning books that teach overfitting usually introduce language model, objective function, generation and accuracy first.
What is the difference between overfitting and underfitting?
Overfitting is a modeling condition in which a model adapts too closely to particular data, including its noise or idiosyncrasies, and consequently generalizes poorly to new data. A modeling failure in which insufficient capacity or training prevents a model from capturing relevant patterns in the data, resulting in poor performance even on training data.
What is overfitting a kind of?
Overfitting is a kind of failure mode. A failure mode is a distinct way in which a system, component, process, or operation can fail or produce an undesirable result.
What is overfitting also called?
Overfitting is also called overfit, over-fitting or overfitted.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
































