Support vector machine

As explained in Machine Learning Algorithms in Depth

A supervised learning model that separates classes by selecting a decision boundary with a large margin, optionally using a kernel for nonlinear separation.

A support vector machine (SVM) is a supervised classification model that chooses a decision boundary with maximum margin. It minimizes the parameter vector’s L2 norm while requiring every training example to be correctly classified. section 5.3 section 5.8

As the norm decreases, the margin expands until further movement would violate the classification constraints. Points on the margin boundaries become support vectors, and these points suffice to determine the learned parameters. section 5.3

The optimization can be developed in several connected steps:

• Formulate the primal problem as minimizing the parameter norm subject to correct-classification constraints. section 5.3
• Construct the dual using a Lagrangian with nonnegative multipliers, turning the primal minimization into a maximization problem. section 5.3
• Use pairwise inner products between training vectors in the dual formulation, then generalize those similarities with a kernel function. section 5.3
• Implement configurable parameters such as C, the kernel, its power, and gamma, while storing multipliers, support vectors, labels, and the intercept. section 5.3

Kernel implementation

A radial-basis-function kernel computes the squared Euclidean distance between two vectors and returns exp(-gamma times that distance), providing a nonlinear similarity measure without explicitly representing transformed features. section 5.3

Figure 5.8 Max margin solution of SVM classifier

Use the maximum-margin diagram to inspect how the decision boundary, margin boundaries, and support vectors relate geometrically. section 5.3

Figure 5.9 Support vector machine pseudo-code

Use the pseudo-code as a compact navigation aid for the SVM training procedure. section 5.3

How Machine Learning Algorithms in Depth covers it

How deep it goes

Derives primal and dual optimization, kernels, geometry, and implementation details, but does not document a complete application workflow.

Examples

Uses margin-boundary geometry, an RBF kernel implementation, pseudo-code, and an Iris classification listing.

What you can do after

Builds an SVM from scratch with configurable kernels and trains it through the described optimization.

How it connects

Connects maximum margin with support vectors, quadratic programming, Lagrangians, duality, inner products, and kernel functions.

liveBookmanning.com

Read Machine Learning Algorithms in Depth on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Grokking Machine LearningA model that separates classes or fits targets using an optimized margin and support vectors.
• 

  liveBookmanning.com
  Sutskever's ListA support vector machine is a supervised learning model that determines decision boundaries using maximum-margin principles.
• 

  liveBookmanning.com
  Data-Driven CybersecurityA support vector machine is a supervised-learning model that separates classes or predicts values by constructing a boundary based on support vectors.
• 

  liveBookmanning.com
  Ensemble Methods for Machine LearningA support vector machine is a supervised learning model that finds a maximum-margin separating function, optionally using kernels to represent nonlinear decision boundaries.

Compare how 5 books explain support vector machine →

Related concepts

• kernel functionuses · A kernel function is a mathematical function that measures similarity between pairs of inputs, often representing an inner product in an implicit feature space.
• convex optimizationuses · Convex optimization is the minimization or maximization of a convex or concave objective over a convex feasible set, a structure that permits global optimality guarantees.
• anomaly detectionuses · Anomaly detection is the identification of observations, events, or behaviors that substantially deviate from expected patterns or a normal baseline.
• classifierkind of · A classifier is a predictive computational model that assigns inputs to one or more discrete classes or categories.

This explainer was generated by Manning's knowledge graph from the text of Machine Learning Algorithms in Depth. The section links point to the parts of the book it draws on.