Constrained optimization selects the best practical solution while satisfying additional requirements. A feasible point meets every constraint; an infeasible point violates at least one. The goal is to maximize the objective among feasible points, even when the unconstrained optimum lies in an unusable region. section 8.0 section 8.6
A constrained optimization workflow typically proceeds as follows:
- Define the objective to maximize and each constraint function, such as model accuracy and computational cost. section 8.0
- Specify constraint bounds; the treatment focuses on inequalities requiring a result y to satisfy a ≤ y ≤ b. section 8.1
- Fit a Gaussian process to the objective and another Gaussian process to each constraint function. section 8.6
- Estimate each candidate's probability of feasibility and combine it with expected improvement to choose the next query. section 8.6
- Track the best observed objective value only among feasible points, using a default when no feasible point has been found. section 8.4
In constrained Bayesian optimization, feasibility changes the search target rather than merely filtering the final answers. A high objective value is useful only when the constraint is likely to hold, so constrained expected improvement balances exploitation of promising objective values with exploration of feasible regions. section 8.6 section 8.4
Airplane design
An airplane-design exercise maximizes flight utility over a four-dimensional unit hypercube while requiring flight cost to remain at or below zero. It compares regular and constrained expected improvement across 50 queries and 10 repeated experiments, then visualizes average progress with error bars. section 8.5
def objective(x): #1
y = -((x + 1) ** 2) * torch.sin(2 * x + 2)
/ 5 + 1 + x / 3 #1
return y #1
def cost(x): #2
return -(0.1 * objective(x) + objective(x - 4))
/ 3 + x / 3 - 0.5 #2
lb = -5 #3
ub = 5 #3
bounds = torch.tensor([[lb], [ub]], dtype=torch.float) #3
Inspect a one-dimensional objective and cost function, together with the bounded search space used to demonstrate constrained expected improvement. section 8.4
