chapter eight
8 The geometry of separation: Vladimir Vapnik and the mathematics of support vector machines
This chapter covers
- Vladimir Vapnik’s The Nature of Statistical Learning Theory (1995) and the emergence of support vector machines (SVMs)
- How margin-based learning reframes model building around generalization rather than fit—a principle that still shapes ML and AI
- How geometric intuition reveals the simplest boundary that separates data
- How that boundary becomes an optimization problem—turning geometry into decision functions, dot products, and slack variables
- How higher-dimensional feature spaces unlock linear separation when patterns appear irreducibly nonlinear
A support vector machine, or SVM, is a learning algorithm that classifies data by drawing the cleanest possible boundary between categories. In its foundational form, an SVM separates two classes, though standard extensions allow for multiclass problems. Rather than memorizing every point in the training data, it identifies the boundary that separates classes while leaving the widest possible gap—called a margin—between them. The points that lie on the edge of this margin are the support vectors, the most informative examples in the data set. At its core, the SVM reframes learning as a problem of generalization: not fitting the training data perfectly, but finding the boundary most likely to perform well on new data.