Linear regression represents a prediction as a weighted combination of input features plus a bias: μ = wᵀx + b. The weights encode each feature’s influence, while the bias shifts the prediction. With one feature such as living area, a positive weight makes predicted housing price rise as area increases. section 3.3.3
As a mean model, linear regression supplies an expected trend before observations are incorporated. A Gaussian process can then model uncertainty around that trend, combining the predictive power of a linear model with uncertainty quantification. A constant mean is the special case w = 0. section 3.2 section 3.3.3
The model is useful when the expected response is plausibly related to features through a linear combination:
- Define the feature vector x, coefficient vector w, and bias b. section 3.3.3
- Fit the coefficients so the resulting line or hyperplane follows the observed targets. section 3.3.3
- Use a linear mean to impose an upward or downward trend, while a covariance model controls additional variability and smoothness. section 3.2 section 3.3.3
Housing prices and structured targets
For housing, living area is the feature and price is the response; a positive coefficient predicts higher prices for larger homes. The same regression framing can treat MNIST digit labels as continuous targets, or predict ad-click probability, molecular energy, or drug effectiveness. section 3.3.3 section 13.1
class LinearMeanGPModel(gpytorch.models.ExactGP):
def __init__(self, train_x, train_y, likelihood):
super().__init__(train_x, train_y, likelihood)
self.mean_module = gpytorch.means.LinearMean(1) #1
self.covar_module = gpytorch.kernels.RBFKernel()
A compact GPyTorch model shows how to replace a constant mean with LinearMean and retain an RBF covariance module. section 3.3.3
