An output scale controls the range of values and predictions produced by a Gaussian process. Increasing it permits larger outputs and more predictive uncertainty; decreasing it narrows the modeled output range. It differs from the length scale: the length scale controls how quickly the function changes across inputs, while the output scale controls how large those changes and values can be. section 3.4.1 section 3.6
Implement the parameter by wrapping a base covariance kernel and train it with the other GP hyperparameters:
- Wrap an RBF or Matérn kernel in GPyTorch's ScaleKernel. section 3.4.1 section 3.5
- Read or set the parameter through model.covar_module.outputscale; the base kernel's lengthscale remains a separate parameter. section 3.4.1
- Optimize the output scale jointly with the model parameters by minimizing negative marginal likelihood with gradient descent. section 3.6 section 12.1.2
Scaling a toy prediction range
With output scale set to 3, the GP models a function with a wider range and allows more uncertainty in its predictions. In contrast, a small output scale produces a narrower output range. This changes amplitude without serving the same role as shortening the length scale, which increases input-wise variability. section 3.4.1 section 3.6
class ScaleGPModel(gpytorch.models.ExactGP):
def __init__(self, train_x, train_y, likelihood):
super().__init__(train_x, train_y, likelihood)
self.mean_module = gpytorch.means.ZeroMean()
self.covar_module =
gpytorch.kernels.ScaleKernel(
gpytorch.kernels.RBFKernel()) #1
def forward(self, x):
... #2
Use the compact model skeleton to see ScaleKernel placed around an RBFKernel, creating the output-scale parameter while retaining the base kernel. section 3.4.1
