liveBook · Manning

Kernel trick

As explained in Timeless Algorithms, The Foundational Ideas

The kernel trick computes inner products in an implicitly transformed feature space without explicitly constructing the transformed feature vectors.

Explicitly expanding features can become impractical as their number grows. The kernel trick replaces dot products in that expanded space with calculations on the original observations, letting an SVM fit a linear hyperplane in an implicit feature space and produce a nonlinear boundary in the original space. section 8.6

K(x, z)=xz+(xz)2

For the mapping ϕ(x) = (x, x²), K(x, z) = xz + (xz)² gives the same value as the dot product ϕ(x) · ϕ(z), without constructing either transformed vector. section 8.6.1

Temperature readings

With x = 3 and z = 6, the transformed vectors are (3, 9) and (6, 36); their dot product is 342. The kernel calculates the same value directly from 3 and 6. section 8.6.1

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Explains the equivalence through a feature map and numerical example, without detailing kernel selection.
Examples
Uses the mapping ϕ(x) = (x, x²) and temperature values 3 and 6 to compare explicit and kernel calculations.
What it assumes
Assumes familiarity with SVM support-vector predictions, dot products, and feature transformations.
What you can do after
Understand how a kernel reproduces transformed-space dot products without explicitly constructing transformed features.
How it connects
Connects support vector machines, support vectors, dot products, feature spaces, and nonlinear decision boundaries.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.