When predictions miss their targets, mean squared error (MSE) summarizes how far off they are. It squares each difference, then averages the squared differences. Squaring makes larger misses count more heavily and keeps positive and negative errors from canceling out. Unlike a total error, an average remains comparable across datasets of different sizes. section 3.23
A small numerical picture
Suppose the errors are 1, -2, and 3. Their squared errors are 1, 4, and 9, so MSE is (1 + 4 + 9) / 3 = 14/3. The 3-unit miss contributes 9, showing why MSE emphasizes larger mistakes. section 3.23
MSE appears in several regression workflows:
- Use sklearn.metrics.mean_squared_error to calculate testing error; taking its square root produces root mean squared error (RMSE). section 4.14
- Train regression decision trees by choosing splits with low MSE. Each group predicts its average label, and the best split has the smallest resulting MSE. section 9.14
- Train regression neural networks with mean squared error as the loss function and an output layer without a final sigmoid activation. section 10.20
MSE is measured in squared target units, so it can be harder to interpret directly. RMSE reverses the squaring through a square root and returns to the original units; an RMSE of $10,000 suggests errors of roughly that scale for house-price predictions. section 3.23
Table 9.7 The nine possible ways to split the dataset by age using a cutoff. Each cutoff splits the dataset into two smaller datasets, and for each of these two, the prediction is given by the average of the labels. The mean square error (MSE) is calculated as the average of the squares of the differences between the labels and the prediction. Notice that the splitting with the smallest MSE is obtained with a cutoff of 35. This gives us the root node in our decision tree.
|
Cutoff
|
Labels left set
|
Labels right set
|
Prediction left set
|
Prediction right set
|
MSE
|
|
0
|
{}
|
{7,5,7,1,2,1,5,4}
|
None
|
4.0
|
5.25
|
|
15
|
{7}
|
{5,7,1,2,1,5,4}
|
7.0
|
3.571
|
3.964
|
|
25
|
{7,5}
|
{7,1,2,1,5,4}
|
6.0
|
3.333
|
3.917
|
|
35
|
{7,5,7}
|
{1,2,1,5,4}
|
6.333
|
2.6
|
1.983
|
|
45
|
{7,5,7,1}
|
{2,1,5,4}
|
5.0
|
3.0
|
4.25
|
|
55
|
{7,5,7,1,2}
|
{1,5,4}
|
4.4
|
3.333
|
4.983
|
|
65
|
{7,5,7,1,2,1}
|
{5,4}
|
3.833
|
4.5
|
5.167
|
|
75
|
{7,5,7,1,2,1,5}
|
{4}
|
4.0
|
4.0
|
5.25
|
|
100
|
{7,5,7,1,2,1,5,4}
|
{}
|
4.0
|
none
|
5.25
|
Compare candidate age cutoffs, group-average predictions, and their MSE values to see how a regression tree selects its root split. section 9.14
















