liveBook · Manning

Preference-based data

As explained in Bayesian Optimization in Action

Preference-based data consists of observations that order or compare alternatives without providing their numerical objective values.

Preference-based data records which alternative is preferred over another, such as f(x₁) > f(x₂), without revealing either alternative’s numerical objective value. This is useful when evaluating an expensive black-box objective is difficult but comparisons remain practical. section 10.3

A Gaussian process can be updated exactly from numerical observations, but conditioning on pairwise comparisons produces a non-GP posterior. A practical approximation therefore selects the GP with the greatest likelihood for the observed comparisons, preserving compatibility with Bayesian optimization policies. section 10.3

from botorch.models.pairwise_gp import PairwiseGP,        #1
 PairwiseLaplaceMarginalLogLikelihood                   #1
from botorch.fit import fit_gpytorch_mll                  #1
 
 
model = PairwiseGP(train_x, train_comp)                   #2
model.covar_module.raw_outputscale.requires_grad_(False)  #3
mll = PairwiseLaplaceMarginalLogLikelihood(model)         #4
fit_gpytorch_mll(mll);                                    #5

Train a preference model with BoTorch’s PairwiseGP, construct its Laplace marginal log likelihood, and fit the model. section 10.3

How Bayesian Optimization in Action covers it

How deep it goes
Defines pairwise comparisons, explains the non-GP posterior consequence, and outlines likelihood-based GP approximation without deriving the approximation.
Examples
Uses comparisons such as f(−0.0374) > f(2.6822) and coded training pairs among 0, 3, −4, and 4.
What it assumes
Assumes familiarity with Gaussian processes, Bayesian updating, likelihood, and Bayesian optimization while contrasting numerical observations with comparisons.
What you can do after
Builds and fits a pairwise preference GP using BoTorch’s PairwiseGP and PairwiseLaplaceMarginalLogLikelihood.
How it connects
Connects pairwise comparisons with Gaussian processes, Bayesian updating, posterior distributions, and Bayesian optimization.
Cover of Bayesian Optimization in Action
liveBookmanning.com
Read Bayesian Optimization in Action on liveBook Get the book at manning.com

This explainer was generated by Manning's knowledge graph from the text of Bayesian Optimization in Action. The section links point to the parts of the book it draws on.