liveBook · Manning

Test error

Test error is the frequency or magnitude of a model’s prediction errors on data held out from fitting for evaluation.

How 2 Manning books explain it

Grokking Machine Learning

Testing error is a measure of a model's prediction error on data held out from training for evaluation.

How it is used
Uses errors on held-out examples to compare models and choose an appropriate model complexity.
How deep it goes
Defines testing error and illustrates MAE and RMSE comparisons, without deriving a general evaluation procedure.
Examples
Compares polynomial models of degrees 1, 2, and 10, and reports testing RMSE for regularized polynomial models.
What you can do after
Evaluate models by comparing their testing errors and identifying evidence of underfitting or overfitting.

Read how Grokking Machine Learning explains test error →

Grokking Machine Learning, Second Edition

A measure of a model's prediction errors on examples reserved for evaluating generalization.

How it is used
Uses held-out testing examples to judge model quality and choose an appropriate level of complexity.
How deep it goes
Explains test error through training-versus-testing comparisons and mean absolute error, without developing computational internals.
Examples
Illustrates testing error with polynomial models of degrees 1, 2, and 10, plus a model-comparison table.
What you can do after
Evaluate model complexity by comparing training and testing errors to identify underfitting, overfitting, or a suitable model.

Read how Grokking Machine Learning, Second Edition explains test error →

Also covered in

What to know first

Related concepts

Questions about test error

What is test error?

Test error is the frequency or magnitude of a model’s prediction errors on data held out from fitting for evaluation.

Which Manning books explain test error?

Grokking Machine Learning and Grokking Machine Learning, Second Edition.

What should I know before learning test error?

Manning books that teach test error usually introduce language model, mean absolute error and polynomial regression first.

What is the difference between test error and objective function?

Test error is the frequency or magnitude of a model’s prediction errors on data held out from fitting for evaluation. An objective function is a mathematical function that assigns values to candidate solutions or parameter settings, which an optimization procedure seeks to minimize or maximize.

What is test error a kind of?

Test error is a kind of error. Prediction error is the discrepancy between a model’s predicted outcome and the corresponding actual or expected outcome.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.