A constraint is an equality or inequality condition that candidate solutions must satisfy. Constrained optimization maximizes an objective only among feasible points; violating points are infeasible. This matters when the unconstrained optimum is too costly, dangerous, or impractical to use. section 8.0 section 8.1.1
Constraints change optimization in several important ways:
- They can cut off a region containing the unconstrained global optimum, lowering the best achievable objective value. section 8.1.1
- They can exclude one optimum and make a different local optimum the best feasible solution. section 8.1.1
- In black-box BayesOpt, each unknown constraint function receives its own Gaussian process, alongside the objective-function model. section 8.1.2
- Constraint-aware acquisition policies combine objective improvement with the probability that a candidate is feasible. section 8.6
A nonpositive-cost constraint
Suppose a one-dimensional objective is maximized while a cost function must satisfy c ≤ 0. The feasible regions are those with nonpositive cost. If the objective's highest values occur where x > 4, the constraint removes that region and reduces the best feasible value to roughly 4. In a related case, ruling out x > 3 shifts the solution to another local optimum. section 8.1.1
A constrained BayesOpt loop retrains the objective GP and one GP per constraint using observed values, then selects the next query with a constraint-aware policy. BoTorch's ConstrainedExpectedImprovement accepts these models, the best feasible objective value, and constraint bounds. section 8.1.2 section 8.4
Figure 8.1 An example of a one-dimensional constrained optimization problem. The solid line is the objective function we aim to maximize, and the dashed line is the cost function that constrains the optimization problem. Only the shaded regions (right panel) that yield negative costs are feasible. Here, the constraint of nonpositive cost causes the highest objective value to decrease from more than 8 to 4.
Use the figure to compare the objective, cost function, feasible shaded regions, and the constrained versus unconstrained optimum. section 8.1.1
