Support vector machine

As explained in Ensemble Methods for Machine Learning

A support vector machine is a supervised learning model that finds a maximum-margin separating function, optionally using kernels to represent nonlinear decision boundaries.

A support vector machine (SVM) is a supervised model for classification and regression. It provides a strong off-the-shelf baseline and can represent nonlinear relationships with kernels such as the radial basis function (RBF) and polynomial kernels. section 1.3 section 1.3.2

SVM training balances fit against regularization. The parameter C controls that balance: small values favor flatter, simpler models that may underfit, while large values prioritize training error and can produce complex models that overfit. section 1.3.2

Use the support-vector view carefully:

• Support vectors are the smaller working set of training examples on which the model depends. section 1.3.2
• The number of support vectors does not directly measure model complexity, because restrictive small C values can force the final model to use more vectors. section 1.3.2
• Compare training and test performance across C values rather than selecting a model from training fit alone. section 1.3.2

Tuning an SVR

With scikit-learn, create an RBF-kernel SVR and evaluate C across a logarithmic range using a validation curve and R² scores. On Friedman-1 regression data, increasing C improves training fit but can reduce generalization. section 1.3.2

Figure 1.5 Support vector machine with an RBF kernel, with kernel parameter gamma = 0.75. Small values of C result in more linear (flatter) and less complex models that underfit the data, while large values of C result in more nonlinear (curvier) and more complex models that overfit the data. Selecting a good value for C is critically important in training a good SVM model.

Visualizes how C changes an RBF-kernel SVM from flatter, more linear fits toward curvier, more complex fits. section 1.3.2

How Ensemble Methods for Machine Learning covers it

How deep it goes

Explains kernels, support vectors, C, validation, and overfitting, but stops short of optimization internals.

Examples

Uses synthetic supervised regression, Friedman-1 data, RBF-kernel SVR code, validation curves, and complexity figures.

What it assumes

Assumes familiarity with labeled supervised data, training and evaluation, and the distinction between fitting and generalization.

What you can do after

Apply it via scikit-learn, choose kernels, tune C, and diagnose overfitting through training and test scores.

How it connects

Connects kernels, regularization, training error, underfitting, overfitting, support vectors, and model validation.

liveBookmanning.com

Read Ensemble Methods for Machine Learning on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Grokking Machine LearningA model that separates classes or fits targets using an optimized margin and support vectors.
• 

  liveBookmanning.com
  Sutskever's ListA support vector machine is a supervised learning model that determines decision boundaries using maximum-margin principles.
• 

  liveBookmanning.com
  Machine Learning Algorithms in DepthA supervised learning model that separates classes by selecting a decision boundary with a large margin, optionally using a kernel for nonlinear separation.
• 

  liveBookmanning.com
  Data-Driven CybersecurityA support vector machine is a supervised-learning model that separates classes or predicts values by constructing a boundary based on support vectors.

Compare how 5 books explain support vector machine →

Related concepts

• kernel functionuses · A kernel function is a mathematical function that measures similarity between pairs of inputs, often representing an inner product in an implicit feature space.
• convex optimizationuses · Convex optimization is the minimization or maximization of a convex or concave objective over a convex feasible set, a structure that permits global optimality guarantees.
• anomaly detectionuses · Anomaly detection is the identification of observations, events, or behaviors that substantially deviate from expected patterns or a normal baseline.
• classifierkind of · A classifier is a predictive computational model that assigns inputs to one or more discrete classes or categories.

This explainer was generated by Manning's knowledge graph from the text of Ensemble Methods for Machine Learning. The section links point to the parts of the book it draws on.