Several boundaries can separate two classes in a training sample, but they need not be equally stable on new observations. A hard-margin support vector machine (SVM) selects the separating hyperplane with the widest margin, while requiring every training observation to be correctly classified and on or beyond its class’s margin boundary. The larger buffer offers a basis for generalization, not a guarantee of it. section 8.1 section 8.3.1
A temperature threshold
Suppose safe and unsafe temperature readings are separable on a single axis. The maximum-margin threshold lies midway between the nearest safe and unsafe readings. These two readings are support vectors: they determine the threshold and the width of its buffer. section 8.1.1
For a feature vector x, the decision function is f(x) = wᵀx + b, where w is the weight vector and b is the bias. The decision boundary is f(x) = 0; the sign of f(x) determines the predicted class. Because multiplying w and b by the same positive constant leaves classifications unchanged, their scale is fixed so that the nearest training observations have values +1 or −1. These values define parallel margin boundaries. section 8.2.2
For training pairs (xᵢ, yᵢ), with labels yᵢ ∈ {−1, +1}, the hard-margin problem minimizes ½‖w‖² subject to yᵢ(wᵀxᵢ + b) ≥ 1 for every i. Each margin boundary lies 1/‖w‖ from the decision boundary, so the total margin width is 2/‖w‖. Minimizing the squared norm therefore maximizes that width while enforcing correct separation. section 8.2.3
Figure 8.4 The dashed line represents the optimized decision boundary, while the two solid lines represent the parallel hard-margin boundaries. The support vectors lie on those margins and constrain their maximum separation. All remaining training observations lie beyond the margins.

The central boundary, parallel margins, and support vectors make the optimization’s geometry visible. section 8.2.3
Support vectors satisfy yᵢ(wᵀxᵢ + b) = 1 and lie on the margin boundaries; observations farther away do not directly constrain the fitted boundary. The objective is convex and the constraints are linear, so, when the training data is linearly separable, optimization yields a globally optimal maximum-margin separator. A new observation may fall inside the margin and is still classified by the sign of its decision function. section 8.2.3 section 8.2.2
Hard constraints make the model sensitive to atypical observations. A point near the opposing class can become a support vector, tilt the boundary, and narrow the margin; overlap or an incorrect label can make the constraints infeasible altogether. A soft-margin SVM instead permits controlled violations, trading training fit against margin width. section 8.3.1 section 8.7.1
