A model complexity graph plots candidate models’ complexity against their training and validation errors. It helps compare models and choose a complexity that avoids both underfitting and overfitting. section 4.4
Read the curves to spot the trade-off:
- As complexity increases, training error generally falls. Validation error often falls at first, then rises. section 4.4
- Choose the candidate with the lowest validation error. Small models may underfit; highly complex models may overfit. section 4.4
Choosing a polynomial degree
Compare polynomial regression models with degrees from 0 to 10. In the illustrated dataset, validation error is lowest at degree 4, making it the preferred candidate among those tested. section 4.4
Figure 4.5 The model complexity graph is an effective tool to help us determine the ideal complexity of a model to avoid underfitting and overfitting. In this model complexity graph, the horizontal axis represents the degree of several polynomial regression models, from 0 to 10 (i.e., the complexity of the model). The vertical axis represents the error, which in this case is given by the mean absolute error. Notice that the training error starts large and decreases as we move to the right. This is because the more complex our model is, the better it can fit the training data. The validation error, however, starts large, then decreases, and then increases again—very simple models can’t fit our data well (they underfit), whereas very complex models fit our training data but not our validation data because they overfit. A happy point in the middle is where our model neither underfits or overfits, and we can find it using the model complexity graph.
The graph illustrates how training error decreases while validation error first decreases and then increases as polynomial degree grows. section 4.4
