The predictive model plays a crucial role in BayesOpt by guiding decision-making with accurate predictions. As we saw in section 1.2.1 and see again and again in this part, GPs offer calibrated quantification of uncertainty, which is a key component in any decision-making task and a feature that many ML models lack.
We begin with chapter 2, which explains the intuition behind a GP as a distribution over functions as well as a generalization of a multivariate normal distribution in infinite dimensions. We explore how via Bayes’ theorem, a GP can be updated to reflect our belief about a function’s value in light of new data.
Chapter 3 showcases the mathematical flexibility of GPs. This flexibility allows us, the users, to incorporate prior information into the predictions via the global trend and the variability of the GP’s predictions. By combining different components of the GP, we gain the ability to mathematically model a wide range of functions.