Test error
Test error is the frequency or magnitude of a model’s prediction errors on data held out from fitting for evaluation.
Grokking Machine Learning
Testing error is a measure of a model's prediction error on data held out from training for evaluation.
- How it is used
- Uses errors on held-out examples to compare models and choose an appropriate model complexity.
- How deep it goes
- Defines testing error and illustrates MAE and RMSE comparisons, without deriving a general evaluation procedure.
- Examples
- Compares polynomial models of degrees 1, 2, and 10, and reports testing RMSE for regularized polynomial models.
- What you can do after
- Evaluate models by comparing their testing errors and identifying evidence of underfitting or overfitting.
Grokking Machine Learning, Second Edition
A measure of a model's prediction errors on examples reserved for evaluating generalization.
- How it is used
- Uses held-out testing examples to judge model quality and choose an appropriate level of complexity.
- How deep it goes
- Explains test error through training-versus-testing comparisons and mean absolute error, without developing computational internals.
- Examples
- Illustrates testing error with polynomial models of degrees 1, 2, and 10, plus a model-comparison table.
- What you can do after
- Evaluate model complexity by comparing training and testing errors to identify underfitting, overfitting, or a suitable model.
Read how Grokking Machine Learning, Second Edition explains test error →
Also covered in
What to know first
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- mean absolute errorMean absolute error is the arithmetic mean of the absolute differences between corresponding predicted and observed numerical values.
- polynomial regressionPolynomial regression is a regression method that models an outcome as a polynomial function of one or more input variables.
Related concepts
- double descentmeasured by · Double descent is a machine-learning performance pattern in which test error decreases, rises near an interpolation threshold, and then decreases again as model capacity or another resource increases.
- permutation feature importancemeasured by · Permutation feature importance measures a feature’s importance to a model by randomly permuting its values and assessing the resulting change in predictive performance.
- errorkind of · Prediction error is the discrepancy between a model’s predicted outcome and the corresponding actual or expected outcome.
- objective functioncontrasts with · An objective function is a mathematical function that assigns values to candidate solutions or parameter settings, which an optimization procedure seeks to minimize or maximize.
Questions about test error
What is test error?
Test error is the frequency or magnitude of a model’s prediction errors on data held out from fitting for evaluation.
Which Manning books explain test error?
Grokking Machine Learning and Grokking Machine Learning, Second Edition.
What should I know before learning test error?
Manning books that teach test error usually introduce language model, mean absolute error and polynomial regression first.
What is the difference between test error and objective function?
Test error is the frequency or magnitude of a model’s prediction errors on data held out from fitting for evaluation. An objective function is a mathematical function that assigns values to candidate solutions or parameter settings, which an optimization procedure seeks to minimize or maximize.
What is test error a kind of?
Test error is a kind of error. Prediction error is the discrepancy between a model’s predicted outcome and the corresponding actual or expected outcome.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.

