A support vector machine (SVM) is a supervised learning model that seeks a class-separating boundary designed to generalize beyond its training observations. Rather than choosing just any boundary that fits the data, it favors one with a wide margin: a buffer between the boundary and nearby observations. The foundational formulation is binary, although extensions handle more than two classes. section 8.1 section 8.1.1
The optimal-margin classifier introduced by Vladimir Vapnik, Bernhard Boser, and Isabelle Guyon in 1992 put this geometric idea into practice. In 1995, Corinna Cortes and Vapnik extended it with a soft margin, allowing violations when a perfect separation would produce a less stable boundary. section 8.0
For a linear SVM, the decision function is f(x) = wᵀx + b, where x is an observation, w is a weight vector, and b is a bias. The sign of f(x) determines the predicted class; f(x) = 0 defines the separating hyperplane. In a hard-margin SVM, the parallel boundaries f(x) = +1 and f(x) = −1 pass through the closest training observations, called support vectors. These observations constrain the fitted boundary; sufficiently distant observations do not directly determine it. section 8.2.2
Figure 8.1 Temperature readings appear along a single horizontal axis, increasing from left to right. Circles represent safe conditions, and squares represent unsafe conditions. The vertical dashed line is the decision boundary. The nearest observation on each side is a support vector, and the distance between the boundary and these points defines the margin on either side. Positioning the boundary midway between the support vectors produces the maximum-margin separator.

A one-dimensional temperature example makes the maximum-margin boundary and its two support vectors visible. section 8.1.1
A hard margin requires linearly separable training data and permits no observations inside the margin. A soft margin accommodates noise, overlap, and unusual observations: slack variables quantify violations, while the penalty parameter C balances reducing them against preserving a wide margin. Accepting some violations can yield a more stable boundary than forcing every training observation to fit. section 8.9 section 8.7.1
Kernels extend SVMs to patterns that are nonlinear in the original feature space. The kernel trick computes dot products in a transformed feature space without explicitly constructing every new feature. The choice of kernel and its parameters controls the kinds and flexibility of available boundaries. section 8.9
A wide margin and controlled flexibility support generalization, but do not guarantee accurate predictions on new data. Cross-validation estimates how configurations may perform on withheld data; an untouched test set evaluates the selected model. Unlike a random forest, which reduces variability by aggregating diverse trees, an SVM seeks stability through constraints on one fitted model. section 8.7 section 9.7






