Polynomial kernel

As explained in Grokking Machine Learning

A kernel function that computes polynomial similarities between input examples and enables polynomial decision regions.

How can an SVM draw a circle when it can separate points only with a line? A polynomial kernel adds polynomial features so a nonlinear dataset becomes linearly separable in a higher-dimensional space. For two inputs, degree-2 features can include x1², x1x2, and x2²; a plane there can project back to a circular, parabolic, or hyperbolic boundary. section 11.3 section 11.4

Follow the geometric idea:

• Start with points that a line cannot separate. section 11.3
• Add columns representing polynomial terms, such as x1², x1x2, and x2². section 11.3
• Separate the expanded points with a plane, then view its projected boundary in the original inputs. section 11.3
• Choose the polynomial degree to model regions shaped by polynomial equations; compare this with the RBF kernel for more complex curved regions. section 11.4

Classifying a noisy circle

A circular dataset cannot be separated well by a line, but the relation x1² + x2² = 1 describes a circle. Fit a degree-2 polynomial SVM with `svm_degree_2 = SVC(kernel='poly', degree=2)`, then inspect the resulting curved classifier. section 11.3

Figure 11.14 Left: The set is not separable by a line. Middle: We look at it in three dimensions, and proceed to raise the two triangles and lower the two squares. Right: Our new dataset is now separable by a plane. (Source: Image created with the assistance of Grapher™ from Golden Software, LLC; https://www.goldensoftware.com/products/grapher).

Visualizes how adding a feature lifts a nonseparable planar dataset into three dimensions, where a plane can separate it. section 11.3

How Grokking Machine Learning covers it

How deep it goes

Explains feature expansion geometrically, works through circular and modified XOR datasets, and demonstrates degree-2 implementation.

Examples

Uses circular points, degree-2 monomial columns, the equation x1² + x2² = 1, and a noisy circular SVC dataset.

What you can do after

Apply it via SVC by selecting `kernel='poly'` and a polynomial degree for nonlinear classification.

How it connects

Connects polynomial kernels to SVMs, kernel methods, feature maps, higher-dimensional spaces, linear classifiers, and RBF kernels.

liveBookmanning.com

Read Grokking Machine Learning on liveBook Get the book at manning.com

Related concepts

• kernel functionkind of · A kernel function is a mathematical function that measures similarity between pairs of inputs, often representing an inner product in an implicit feature space.
• polynomial degreeconfigured by · The degree of a polynomial is the greatest exponent with a nonzero coefficient.
• kernel methoduses · A kernel method is a machine-learning technique that uses a kernel function to model relationships in a feature space without explicitly computing the feature representations.

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning. The section links point to the parts of the book it draws on.