A model can fit its training observations closely and still fail on new ones. Generalization is its ability to perform well on observations not used during training. Training accuracy alone cannot establish that the relationships a model has learned will persist beyond its sample. section 8.0 section 13.3.2
Model capacity helps determine which relationships a model can fit. Too much flexibility can allow it to capture noise or sample-specific details; capacity alone, however, does not determine the outcome. Statistical learning theory relates observed error, capacity, and sample size to the likelihood that learned structure will persist beyond the training data. section 8.7 section 11.8
Different methods pursue generalization in different ways:
- Support vector machines control the flexibility of a single classifier. A wide margin reduces sensitivity to the training sample; a soft margin permits violations when accommodating unusual observations leads to a more stable boundary. section 8.0 section 8.7
- Random forests combine flexible decision trees. Aggregation improves stability when trees are individually useful but do not make highly correlated errors; diversity without predictive strength is not enough. section 9.4 section 9.7
- Neural networks can learn reusable representations. Weight decay, dropout, early stopping, and data augmentation constrain how network capacity is used, but greater capacity alone does not guarantee generalization. section 11.5.1 section 11.8
A temperature threshold
Suppose safe and unsafe temperature readings can be separated by several thresholds. The maximum-margin threshold lies midway between the closest readings on opposite sides. Its buffer makes the boundary less sensitive to small changes in the sample and more likely to classify new readings reliably, though success on future readings is not guaranteed. section 8.1.1 section 8.7
Figure 8.1 Temperature readings appear along a single horizontal axis, increasing from left to right. Circles represent safe conditions, and squares represent unsafe conditions. The vertical dashed line is the decision boundary. The nearest observation on each side is a support vector, and the distance between the boundary and these points defines the margin on either side. Positioning the boundary midway between the support vectors produces the maximum-margin separator.

The figure shows the nearest readings, the decision boundary, and the margin on each side. section 8.1.1
Cross-validation estimates how configurations may perform on withheld data; an untouched test set assesses the selected model on new observations. A test result cannot improve an already fitted boundary without revisiting training or model selection. In random forests, out-of-bag observations provide another performance estimate, while an independent test set remains valuable after tuning. section 8.7 section 9.9
Evidence of generalization is conditional on the task and data evaluated. Large language models have shown declining loss on held-out text as parameters, data, and compute increased, without establishing reliability on every unfamiliar task or distribution. In deployment, changes in conditions can require continued monitoring; predictive accuracy also leaves questions about fairness and consequences unresolved. section 13.3.2 section 1.2.6


















