chapter six

6 Leveraging information theory with entropy-based policies

 

This chapter covers

  • Entropy as an information theoretic measure of uncertainty
  • Information gain as a method of reducing entropy
  • Bayesian optimization policies that leverage information theory for their search

We saw in Chapter 4 that by aiming to improve from the best value that we have seen so far, we can design improvement-based Bayesian optimization (BayesOpt) policies such as Probability of Improvement and Expected Improvement. In Chapter 5, we leveraged multi-armed bandit policies to obtain Upper Confidence Bound and Thompson sampling, each of which uses a unique heuristic to balance between exploration and exploitation in the search for the global optimum of the objective function.

6.1 Measuring knowledge with information theory

6.1.1 Measuring uncertainty with entropy

6.1.2 Looking for a remote control using entropy

6.1.3 Binary search using entropy

6.2 Entropy search in Bayesian optimization

6.2.1 Searching for the optimum using information theory

6.2.2 Implementing entropy search with BoTorch

6.3 Summary

6.4 Exercise

6.4.2 Bayesian optimization for hyperparameter tuning