11 Optimizing multiple objectives at the same time
"This coffee tastes good, but there’s too much sugar." "That shirt looks great, but it’s out of my price range." "The neural network I just trained has a high accuracy, but it is too big and takes too long to train."
Every day, we are faced with optimization trade-offs of the same flavor as those above where, in an attempt to achieve a good performance on some objective, we sacrifice another criterion that’s just as important: a coffee drinker with a sweet tooth might optimize the taste of their coffee while using an unhealthy amount of sugar; a shopper scores a clothing article high on looks and low on affordability; a machine learning engineer develops a neural network that has good predictive performance but is too large to be used in a real time application.
By focusing on one optimization objective, we might perform badly on another objective that needs to be accounted for. We should instead model all objective functions that are to be optimized into our optimization procedure, and attempt to jointly optimize them all. For example, we should look for coffee recipes that are both tasty and low on sugar, or clothing items that are both fashionable and affordable, or machine learning models that perform well and are practical to implement. This type of optimization problem is called multi-objective optimization.