- In chapter 2 (A/B testing), we talked about how to take a measurement of a business metric.
- In chapters 3 (multi-armed bandits) and 5 (contextual bandits), we saw that if you adapt your experiments based on uncertainty estimates, you can improve your business metric while your experiment is running. We said we were “balancing exploration with exploitation.”
- In chapter 4 (response surface methodology), we showed how to use estimates of a business metric—a surrogate function—to reduce the number of measurements required to optimize parameters.