A hyperparameter is a control value chosen to define or run a model, rather than a learned model weight. Tuning hyperparameters means changing those controls to improve a model or optimization process. The work is practical but consequential: poorly tuned options can leave a deep-learning model seriously flawed. section 1.5.3 section 5.1
Common model controls include learning rate, epochs, middle-layer size, and batch size. Batch size sets how many inputs enter a forward pass; a data-step control can change the amount of generated training data. These choices interact: increasing data by reducing data_step may require a larger batch size, while a larger middle layer can fit the target better but take longer to train. section 5.1.1
Evolutionary algorithms have hyperparameters of their own:
- Population and generations set the number of individuals simulated and the number of evolutionary iterations; both relate to chromosome complexity and can be balanced against each other. section 2.5.9
- Crossover rate controls whether, how often, or where parents share genes. section 2.5.9
- Mutation rate trades population variety against specialization: high mutation can help complex problems but can also obstruct optimum performance. section 2.5.9
Instead of selecting values entirely by hand, evolutionary computation can optimize hyperparameters across deep-learning models using particle swarm optimization, differential evolution, or genetic algorithms. Encoding only hyperparameters produces a smaller search space than encoding an entire architecture, which matters when complex GAN training may take hours or days. section 1.5.3 section 9.4
Encoding DCGAN controls as genes
Represent each model variation as an array of floats. Map fixed gene positions into bounded values for filter count, alpha, critic iterations, clipping, and learning rate, then pass the decoded values into the DCGAN initializer. This lets an evolutionary optimizer search model settings while the network is trained and scored. section 9.4
Listing 5.1 EDL_5_1_Hyperparameter_Tuning.ipynb: Hyperparameters
hp_test = "test 1" #@param {type:"string"} #1
learning_rate = 3.5e-03
epochs = 500
middle_layer = 25
batch_size = 2 #2
data_step = 1 #3
Inspect a runnable configuration block containing learning_rate, epochs, middle_layer, batch_size, and data_step before changing one setting and rerunning the remaining cells. section 5.1.1
