2 Gaussian processes as distributions over functions

Having seen what BayesOpt can help us do, we are now ready to embark on our journey toward mastering BayesOpt. As we saw in chapter 1, a BayesOpt workflow consists of two main parts: a Gaussian process (GP) as a predictive, or surrogate, model and a policy for decision-making. With a GP, we don’t obtain only point estimates as predictions for a test data point, but instead, we have an entire probability distribution representing our belief about the prediction.

With a GP, we produce similar predictions from similar data points. For example, in weather forecasting, when estimating today’s temperature, a GP will look at the climatic data of days that are similar to today, either the last few days or this exact day a year ago. Days in another season wouldn’t inform the GP when making this prediction. Similarly, when predicting the price of a house, a GP will say that similar houses in the same neighborhood as the prediction target are more informative than houses in another state.

2.1 How to sell your house the Bayesian way

2.2 Modeling correlations with multivariate Gaussian distributions and Bayesian updates

2.2.1 Using multivariate Gaussian distributions to jointly model multiple variables

2.2.2 Updating MVN distributions

2.2.3 Modeling many variables with high-dimensional Gaussian distributions

2.3 Going from a finite to an infinite Gaussian

2.4 Implementing GPs in Python

2.4.1 Setting up the training data

2.4.2 Implementing a GP class

2.4.3 Making predictions with a GP

2.4.4 Visualizing predictions of a GP

2.4.5 Going beyond one-dimensional objective functions

2.5 Exercise

Summary