An evaluation budget caps the cumulative cost of objective-function evaluations during an optimization run. In multifidelity optimization, that budget can purchase either low- or high-fidelity queries. section 9.4
A budget-based optimization loop:
- Sets a limit such as 10 cost units. section 9.4
- Continues while cumulative spending remains below the limit. section 9.4
- Adds the selected query's fidelity-dependent cost after each iteration. section 9.4
- Records recommendations against spent budget rather than against query count. section 9.4
Multifidelity BayesOpt
A loop with budget_limit = 10 may alternate inexpensive low-fidelity evaluations with costly high-fidelity evaluations, stopping when cumulative cost exceeds 10. Performance is plotted as objective value against budget spent. section 9.4
Figure 9.13 Measuring performance with the high-fidelity incumbent is inappropriate in multifidelity optimization. On the left, the high-fidelity incumbent is roughly 0.8, while we haven’t discovered the objective’s optimum. On the right, even though we are close to locating the objective’s optimum, there’s no high-fidelity query to record the incumbent value with.
Contrasts misleading high-fidelity-incumbent measurements with progress measured through a final recommendation. section 9.4
