Bayesian optimization sequentially optimizes an expensive, noisy black-box objective by measuring configurations, updating a probabilistic surrogate, and selecting the next experiment through an acquisition function. It reduces costly measurements while balancing exploration of uncertainty against exploitation of promising results. section 6.0 section 6.1.1
A practical optimization loop proceeds as follows:
- Choose an initial configuration and measure the objective. section 6.1.2
- Fit a response-surface surrogate with Gaussian process regression. section 6.0 section 6.5
- Optimize an acquisition function to select the next configuration, combining expected improvement with information gathering. section 6.1.5 section 6.5
- Repeat measurement, modeling, and selection until the model and proposed configuration stop changing. section 6.1.4
Optimizing a JIT compiler
Treat seven compiler parameters as a black box and measure average CPU time. A one-parameter walkthrough starts at 0.5, records approximately 1.56 CPU-time units, then measures 0, recording approximately 1.20. The surrogate favors uncertain boundary values for exploration, low predicted CPU time for exploitation, or an intermediate value such as 0.11 when both goals are combined. section 6.1.2 section 6.1.4 section 6.1.5
Listing 6.1 JIT CPU time simulator
def jit_plus_server(parameters):
x = np.array(parameters) #1
d = len(x)
x1 = x - 0.15*np.ones(shape=(d,)) #2
x2 = x - 0.85*np.ones(shape=(d,)) #2
cpu_time = 2 - np.exp(-10*x1**2) - 0.5*np.exp(-10*x2**2) #2
return cpu_time.mean() + .005*np.random.normal() #3
Inspect the noisy JIT CPU-time simulator used as the black-box objective; it accepts parameter values and returns a single measured CPU-time value. section 6.1.1
