Neuroevolution applies evolutionary methods to deep-learning optimization. It can search hyperparameters, network weights and parameters, or network architecture, replacing or extending conventional training choices. Parameter-focused neuroevolution assigns a candidate gene sequence to a network’s weight tensors and evaluates the resulting model rather than backpropagating loss through an optimizer. section 6.0 section 6.2
Compare the two optimization paths: backpropagation propagates calculated loss, while neuroevolution supplies candidate gene values as network weights. section 6.1
A simple implementation starts with an MLP whose train and back-propagation functions are removed. For each candidate individual, split its flat gene sequence according to every parameter tensor’s size, reshape each slice, and install the reshaped tensors as the model’s weights. Fitness comes from the network’s predictions, allowing a genetic algorithm to select, recombine, and mutate weight assignments. section 6.2
A practical weight-search loop is:
- Build a network and count all trainable parameter values; that count becomes the individual’s gene length. section 6.2 section 6.4
- Initialize genes over a chosen range, then configure selection, crossover, and mutation in DEAP. section 6.2
- Install an individual’s genes as weights, predict on the dataset, and return a fitness based on classification accuracy. section 6.2
- Evolve populations across generations, inspect current and best predictions, and retain the best individual’s weights. section 6.2
Evolving an MLP for circles
Use circles or moons datasets to compare optimization methods. A NumPy MLP provides a transparent baseline; a genetic algorithm can evolve its weights, while evolutionary strategies and differential evolution offer alternative searches. In the demonstrated sample problems, the simpler GA generally performs best, though results may vary and differential evolution is described as less optimal there. section 6.1 section 6.3
The same gene-to-tensor idea transfers to Keras. Count trainable parameters, flatten candidate weight values into an individual, reconstruct each layer’s weight arrays in their original shapes, and call set_weights. This makes a direct comparison possible between GA optimization and a Keras model trained with Adam and binary cross-entropy. section 6.4
Neuroevolution can also optimize architecture. For convolutional networks, a custom genetic sequence can encode layer type, number, kernel size, stride, normalization, and activation function; crossover and mutation must then support that structured encoding. Such architecture search can aid complex CNN design but may limit the number of layers supported by the encoding. section 7.6
Experiment by changing dataset difficulty, hidden layers, added or removed Keras layers, and loss functions such as mean squared error versus binary cross-entropy. These variations expose practical limits of evolutionary search rather than guaranteeing improvement for every network or problem. section 6.1 section 6.4.1
