A pairwise comparison records which of two candidate inputs is preferred: rather than asking for an exact score, ask which item the user likes better. This can reduce the cognitive burden of reporting preferences, making feedback more consistent with a user’s underlying preference when ratings are noisy or difficult to provide. The trade-off is that one comparison contains less information than a numerical evaluation. section 10.0 section 10.1 section 10.5
Pairwise comparisons are especially useful when a decision combines criteria whose relative weights are hard to specify. Instead of turning appearance, cost, practicality, and efficiency into one uncertain score, a decision-maker can compare two alternatives head-to-head. In preference optimization, each answer becomes evidence about an unknown objective or preference function. section 10.1 section 10.2
Encoding a comparison
For two evaluated inputs at indices 0 and 1, if input 0 has the greater objective value, encode the outcome as [[0, 1]]: the first index identifies the preferred input and the second identifies the non-preferred input. A collection of these index pairs can train a preference model without retaining numerical ratings as observations. section 10.2 section 10.5
Figure 10.1 Examples of user’s preference elicitation in production recommendation. On the left, the user is asked to rate a recommended product. On the right, the user is asked to pick the product they like better. The latter situation helps better elicit the user’s preference.
Contrast a numerical product rating with choosing the better of two products; the latter directly elicits a relative preference. section 10.0
A preference-based Gaussian process can be fitted to comparison data by approximating the posterior with a GP that has high likelihood for the observed outcomes. Its predicted mean should rank preferred training locations above locations that lost comparisons, while uncertainty decreases only modestly because comparisons are less informative than numerical observations. section 10.3 section 10.5
To choose the next question, form a pair rather than a single evaluation point. A practical “king of the hill” policy compares the current best input with a challenger selected by a Bayesian optimization acquisition policy, reusing its exploration–exploitation behavior to seek an improvement. section 10.4 section 10.5
Finding a customer’s preferred shirt
Treat the number of flowers on a Hawaiian shirt as a one-dimensional input. Present two shirts, record which floral level the customer prefers, and repeat comparisons to locate the level with greatest preference. The example includes a local optimum at a more-than-half floral shirt and a global optimum near a fully floral shirt. section 10.2 section 10.2
