An RBF kernel measures similarity through distance: nearby inputs receive high covariance, while increasingly distant inputs receive lower covariance. Its length scale controls how quickly similarity falls. As a Gaussian-process covariance function, it encodes the assumption that nearby data points tend to behave alike. section 2.4.2 section 13.2
Similarity on the real number line
With GPyTorch, the covariance between 0 and 0.1 is approximately 0.99, whereas the covariance between 0 and 10 is 0. The small difference produces strong similarity; the large difference produces almost none. section 13.2.1
Use the kernel as a covariance module inside a Gaussian-process model:
- Create an RBFKernel, optionally wrapped in a ScaleKernel to provide an output scale. section 2.4.2 section 12.1.2
- Pass inputs through the mean and covariance modules, then combine them into a MultivariateNormal distribution. section 2.4.2
- Train the kernel hyperparameters, including length scale and output scale, by minimizing the negative exact marginal log likelihood with Adam. section 12.1.2
Distance is not always a meaningful measure of similarity. For structured inputs such as images, the RBF kernel can confuse points whose pixel values are numerically close but structurally different, motivating learned representations before applying a kernel. section 13.2 section 13.2.4
class GPModel(gpytorch.models.ExactGP):
def __init__(self, train_x, train_y, likelihood):
super().__init__(train_x, train_y, likelihood)
self.mean_module = gpytorch.means.
ConstantMean() #1
self.covar_module = gpytorch.kernels.
ScaleKernel( #2
gpytorch.kernels.RBFKernel() #2
) #2
def forward(self, x): #3
mean_x = self.mean_module(x) #3
covar_x = self.covar_module(x) #3
return gpytorch.distributions.MultivariateNormal #3
(mean_x, covar_x) #3
Inspect a complete GPyTorch covariance-module definition using a scaled RBF kernel. section 12.1.2
