1 Introduction to Bayesian optimization

I’m very happy that you are reading this book and excited for your upcoming journey. On a high level, Bayesian optimization is an optimization technique that may be applied when the function (or in general any process that gives you an output when an input is passed in) we are trying to optimize is a black box and expensive to evaluate in terms of time, money, or other resources. This setup encompasses many important tasks including hyperparameter tuning (which we will define shortly). Using Bayesian optimization could accelerate this search procedure and help us locate the optimum of the function as quickly as possible.

1.1 Finding the optimum of an expensive, black-box function is a difficult problem

1.1.1 Hyperparameter tuning as an example of an expensive black-box optimization problem

1.1.2 The problem of expensive, black-box optimization

1.1.3 Other real-world examples of expensive black-box optimization problems

1.2 Introducing Bayesian optimization

1.2.1 Modeling with a Gaussian process

1.2.2 Making decisions with a Bayesian optimization policy

1.2.3 Combining the Gaussian process and the optimization policy to form the optimization loop

1.2.4 Bayesian optimization in action

1.3 What will you learn in this book?

1.4 Summary