Regularization is not directly defined here. The related treatment focuses on loss functions as optimization objectives: gradient descent minimizes a GP likelihood, while numerical instability can mislead optimization and produce a suboptimal model. section 12.1.2 section 12.1.3
For scalable Gaussian processes, a variational GP uses inducing points and optimizes the ELBO as a proxy for marginal log likelihood. Mini-batches stabilize loss computation, while natural gradient descent updates variational parameters. section 12.5
GP loss optimization
A GPyTorch model trains with Adam for 500 iterations, computes the negative exact marginal log likelihood as loss, and switches between training and evaluation modes.
optimizer = torch.optim.Adam(model.parameters(), lr=0.01) #1
mll = gpytorch.mlls.ExactMarginalLogLikelihood(likelihood, model) #2
model.train() #3
likelihood.train() #3
for i in tqdm(range(500)): #4
optimizer.zero_grad() #4
output = model(train_x) #4
loss = -mll(output, train_y) #4
loss.backward() #4
optimizer.step() #4
model.eval() #5
likelihood.eval() #5
Shows the complete loss-minimization loop for a regular GP, including the optimizer, objective, gradient computation, parameter update, and evaluation mode. section 12.1.2
