In the last chapter, we managed to get our feet wet by employing evolutionary algorithms for the purposes of optimizing DL network hyperparameters. We saw how using EA could improve the search for hyperparameters beyond simple random or grid search algorithms. Employing variations of EA, such as PSO, evolutionary strategy, and differential evolution, uncovered insights into methods used to search and for hyperparameter optimization (HPO).
Evolutionary DL is a term we use to encompass all evolutionary methods employed to improve DL. More specifically, the term neuroevolution has been used to define specific optimization patterns applied to DL. One such pattern we looked at in the last chapter was the application of evolutionary algorithms to HPO.
Neuroevolution encompasses techniques for HPO, parameter optimization (weight/ parameter search), and network optimization. In this chapter, we dive into how evolutionary methods can be applied to optimize network parameters directly, thus eliminating the need to backpropagate errors or loss through a network.