A kernel function assigns weight to each measurement according to its distance from a query input. In Gaussian process regression, nearby observations influence CPU-time estimates more strongly than distant ones. The common squared exponential kernel uses distance and a length scale, sigma, to control smoothness. section 6.2.1
Practical properties include:
- Normalize the kernel weights before computing a weighted average; otherwise, their total can inflate the estimate. section 6.2.1
- Increase sigma for a smoother surrogate surface; select it from measurements with leave-one-out cross-validation when needed. section 6.2.1
Estimating CPU time
With parameters [0.5, 0.0], measurements [1.52, 1.21], and sigma = 0.25, kernel-weighted estimates at queries 0.25 and 0.4 are 1.365 and 1.448. section 6.2.1
Listing 6.3 GPR with a weighted average
class GPR2:
def __init__(self, parameters, measurements, sigma):
self.x = parameters
self.y = np.array(measurements)
self.sigma = sigma
self.mean_y = self.y.mean()
self.y -= self.mean_y #1
def kernel(self, x1, x2): #2
distance_squared = ((x1-x2)**2).sum()
return np.exp( -distance_squared/(2*self.sigma**2) )
def estimate(self, query_parameter):
weights = [
self.kernel(x, query_parameter)
for x in self.x
] #3
weights = np.array(weights)
weights = weights / weights.sum() #4
return self.mean_y + weights @ self.y #5
Inspect a NumPy-style implementation that computes squared distances, builds kernel weights, normalizes them, and applies the weighted measurement residuals. section 6.2.1
