Radial basis function kernel

As explained in Grokking Machine Learning

A kernel based on a radial function whose value depends on the distance between an input point and a center point.

Suppose a straight line cannot separate triangles from squares. An RBF kernel adds a distance-based similarity around each data point, allowing an SVM to form curved decision boundaries. section 11.3 section 11.4

For points p and q, similarity decreases as their distance increases: similarity(p,q) = e^(−distance(p,q)²). Each point therefore creates a Gaussian-like bump centered at itself. section 11.3

An RBF-kernel classifier works by:

• Embedding each example into a space whose coordinates measure similarity to the data points. section 11.3
• Raising positive examples and lowering negative examples with radial basis functions. section 11.3
• Finding a separating plane in the enhanced space, whose projection becomes a curved boundary in the original space. section 11.3

A one-dimensional classifier

With label-1 points at −1 and 1 and a label-0 point at 0, no single dividing point works. The RBF model creates two bumps and one valley; their combined curve crosses zero twice, classifying the middle interval as label 0 and the outside regions as label 1. section 11.3

Figure 11.20 Using an SVM with the RBF kernel to separate a nonlinear dataset in one dimension. Top: We draw a mountain (radial basis function) at each point with label 1 and a valley at each point of label 0. Bottom: We add the radial basis functions from the top figure. The resulting function intersects the axis twice. The two points of intersection are the boundary of our SVM classifier. We classify each point between them as a square (label 0) and every point outside as a triangle (label 1).

Visualizes the bumps, valleys, summed surface, and two resulting decision boundaries in the one-dimensional example. section 11.3

The gamma parameter controls each bump’s width: small gamma produces a broad surface, while large gamma produces a narrow one. Changing it changes the classifier’s boundary. section 11.3

How Grokking Machine Learning covers it

How deep it goes

Explains similarity features, summed radial functions, projected boundaries, and gamma width, without deriving the full kernel mathematics.

Examples

Uses one-dimensional points, two-dimensional triangle-and-square datasets, similarity tables, circular boundaries, and different gamma values.

What it assumes

Assumes the basic idea of SVM classification and linear separability while motivating nonlinear boundaries geometrically.

What you can do after

Apply it via an SVM to classify datasets with complex curved regions.

How it connects

Connects SVMs, kernel methods, linear classifiers, higher-dimensional spaces, polynomial kernels, and gamma.

liveBookmanning.com

Read Grokking Machine Learning on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Bayesian Optimization in ActionAn RBF kernel is a radial basis function covariance or similarity function whose value decreases with distance between inputs according to a length-scale parameter.
• 

  liveBookmanning.com
  Machine Learning Algorithms in DepthA kernel function whose value typically decreases exponentially with the squared Euclidean distance between its input vectors.

Compare how 3 books explain radial basis function kernel →

Related concepts

• squared exponential kernelequivalent to · A squared exponential kernel is a similarity or covariance function whose value decreases exponentially as the squared distance between inputs increases.
• kernel functionkind of · A kernel function is a mathematical function that measures similarity between pairs of inputs, often representing an inner product in an implicit feature space.
• Gaussian processused by · A Gaussian process is a probability distribution over functions in which every finite collection of function values has a joint Gaussian distribution.
• output scaleused by · An output scale is a covariance amplitude parameter that controls the range of a Gaussian process's function values and predictions.

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning. The section links point to the parts of the book it draws on.