Probability of feasibility measures how likely a candidate is to satisfy an inequality constraint under a predictive distribution. In constrained Bayesian optimization, it turns uncertainty about a constraint function into a usable score: likely-feasible candidates retain priority, while unlikely-feasible candidates are down-ranked. section 8.2 section 8.6
A Gaussian process trained on observed constraint values supplies a normal predictive distribution at each unseen candidate. The probability of feasibility is the portion of that distribution satisfying the constraint; for a cost constraint requiring cost ≤ 0, it is the probability that the predicted cost is below zero. section 8.2 section 8.3
Constrained expected improvement combines feasibility with objective improvement:
- Constrained EI = EI × probability of feasibility. section 8.2
- High expected improvement cannot compensate indefinitely for a very low feasibility probability, so infeasible regions are de-prioritized. section 8.2
- The method balances exploiting promising objective values with staying inside feasible regions. section 8.2 section 8.6
One-dimensional cost constraint
Consider candidates on the interval −5 to 5. Train one Gaussian process on the objective and another on a cost function, then evaluate the cost GP across a dense test grid. At x = 1, the probability of feasibility is the part of the predictive normal distribution lying below zero. Multiplying this probability by each candidate’s EI score produces constrained EI, which can recommend a different point from regular EI when the high-value region is costly. section 8.3
The quantity can be computed manually with PyTorch’s Normal distribution and its CDF evaluated at zero, or delegated to BoTorch’s ConstrainedExpectedImprovement policy. The policy receives objective and constraint Gaussian processes and the constraint bounds. section 8.3 section 8.4 section 8.6
