The squared exponential kernel assigns a measurement a weight that decreases exponentially with the squared distance to a query point. In Gaussian process regression, it is the most common kernel shape for turning nearby measurements into a smooth estimate. section 6.2.1
For scalar inputs, a basic form is weight = exp(−(x − parameter)²). The practical form scales distance by a length scale σ: exp(−((x − parameter)/(2σ))²). Larger σ values make the surrogate smoother, while smaller values make distance matter more sharply. section 6.2.1
A Gaussian process regression implementation uses the kernel to:
- Compute squared distance between an observed parameter and a query parameter. section 6.2.1
- Normalize kernel values so nearby measurements contribute through a weighted average. section 6.2.1
- Build the measurement-to-measurement kernel matrix and combine it with the query kernels for full GPR weights. section 6.2.1
- Treat clustered observations cautiously because several nearby measurements can collectively receive excessive influence. section 6.2.1
CPU-time estimation
With parameters [0.5, 0.0], measurements [1.52, 1.21], and σ = 0.25, the weighted GPR estimates are 1.365 at query 0.25 and 1.448 at query 0.4. The same kernel can generate a smooth CPU-time curve across a parameter range. section 6.2.1
The kernel also supports a relative uncertainty estimate. Certainty is highest at measured parameters and tends toward zero farther from measurements; uncertainty is modeled as one minus squared certainty. This model uncertainty is not comparable across separate regressions like standard error is. section 6.2.2
Listing 6.3 GPR with a weighted average
class GPR2:
def __init__(self, parameters, measurements, sigma):
self.x = parameters
self.y = np.array(measurements)
self.sigma = sigma
self.mean_y = self.y.mean()
self.y -= self.mean_y #1
def kernel(self, x1, x2): #2
distance_squared = ((x1-x2)**2).sum()
return np.exp( -distance_squared/(2*self.sigma**2) )
def estimate(self, query_parameter):
weights = [
self.kernel(x, query_parameter)
for x in self.x
] #3
weights = np.array(weights)
weights = weights / weights.sum() #4
return self.mean_y + weights @ self.y #5
Inspect a compact NumPy implementation that centers measurements, computes squared-exponential kernel values, normalizes weights, and returns the weighted estimate. section 6.2.1
