Squared exponential kernel

As explained in Ensemble Methods for Machine Learning

An exponential smoothing kernel assigns similarity weights that decrease exponentially with the squared distance between two inputs.

A squared-exponential kernel turns distance into similarity: nearby inputs receive large weights, while distant inputs receive rapidly smaller weights. It is useful when a model should emphasize what happens locally around one example. section 9.4.1

For a perturbed point x′ around an example x:

• Compute the distance D(x, x′), such as Euclidean distance. section 9.4.1
• Calculate πx(x′) = exp(−γ · D(x, x′)²). section 9.4.1
• Use the resulting weights to prioritize nearby samples when fitting a local linear surrogate. section 9.4.1

Figure 9.14 LIME creates a surrogate training set of examples in the locality of the example whose prediction needs to be explained. These examples are further weighted by their distance. This is indicated by the sizes of the surrogate examples, with closer examples getting higher weights (and shown larger). A weighted loss function is used to fit a linear surrogate model, which provides local explanations.

Visualizes locality through larger markers for closer samples and a weighted loss for the surrogate model. section 9.4.1

How Ensemble Methods for Machine Learning covers it

How deep it goes

Defines the equation, distance weighting, and width parameter, then applies the kernel to local surrogate fitting.

Examples

Uses perturbed five-feature examples and a visual LIME neighborhood whose closer samples receive larger weights.

What you can do after

Apply it via LIME to emphasize local samples when explaining XGBoost predictions.

How it connects

Connects similarity weighting with LIME, Euclidean distance, linear models, L1 regularization, and radial basis function kernels.

liveBookmanning.com

Read Ensemble Methods for Machine Learning on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Experimentation for EngineersThe squared exponential kernel is a Gaussian-process covariance function whose similarity decreases exponentially with squared input distance.

Compare how 2 books explain squared exponential kernel →

Related concepts

• radial basis function kernelequivalent to · A radial basis function kernel is a similarity function whose value depends on the distance between inputs and typically decreases as that distance increases.
• kernel functionkind of · A kernel function is a mathematical function that measures similarity between pairs of inputs, often representing an inner product in an implicit feature space.

This explainer was generated by Manning's knowledge graph from the text of Ensemble Methods for Machine Learning. The section links point to the parts of the book it draws on.