How deeply does Grokking Machine Learning cover kernel method?

Explains geometric transformations, similarity features, RBF gamma, polynomial kernels, and Scikit-Learn examples, while deferring full kernel theory.

Kernel method

As explained in Grokking Machine Learning

A machine-learning technique that maps data into a higher-dimensional feature space so linear methods can model nonlinear relationships.

A straight line cannot separate every dataset. The kernel method reshapes the problem by adding columns or embedding points in a higher-dimensional space, where a linear boundary may separate them; projecting that boundary back produces a nonlinear classifier. section 11.3

Follow the method as a geometric transformation:

Start with points that a line cannot separate. section 11.3
Add informative columns, such as x1x2, x1², x1x2, and x2², or use similarity columns generated by a kernel. section 11.3
Train a linear classifier, typically an SVM, in the expanded space. section 11.0 section 11.4
Project the boundary back to the original space, where it can appear as a circle, curve, or more complex region. section 11.4

Turning a circle into a linearly separable problem

Suppose triangles and squares form concentric groups. Add the feature x1² + x2², which records each point’s squared distance from the origin. In three dimensions, a plane can separate the groups; projected back onto the original plane, that separation becomes the circular boundary x1² + x2² = 1.

Using an RBF kernel

An RBF kernel creates a similarity feature for each reference point: nearby points receive high similarity and distant points receive low similarity. Combining these features lets an SVM form curved boundaries. Gamma controls the width of each similarity surface: smaller values make it wider, while larger values make it narrower.

How Grokking Machine Learning covers it

liveBook manning.com

Read Grokking Machine Learning on liveBook Get the book at manning.com

Related concepts

spaceuses · A mathematical space is a set of elements equipped with structure or relationships that provide a context for mathematical analysis.
machine learningkind of · Machine learning is a field of computational methods in which systems learn patterns or relationships from data to make predictions, decisions, or other outputs.
support vector machineuses · A support vector machine is a supervised-learning model that uses support vectors to determine a maximum-margin decision boundary or regression function, optionally using kernels to model nonlinear relationships.
radial basis function kernelused by · A radial basis function kernel is a similarity function whose value depends on the distance between inputs and typically decreases as that distance increases.
polynomial kernelused by · A polynomial kernel is a kernel function that measures similarity between inputs using a polynomial of their inner product.
nonlinear classifierproduces · A classifier whose decision boundary cannot be represented by a single linear or affine function of the input features.

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning. The section links point to the parts of the book it draws on.