Neural network size

Q: How deeply does Evolutionary Deep Learning: Genetic algorithms and neural networks cover neural network size?

Defines layers and width, gives tuning rules and experiments, but does not derive network-capacity theory.

As explained in Evolutionary Deep Learning: Genetic algorithms and neural networks

Network size is the scale of a neural network as determined by its numbers of nodes and layers.

Network size describes a neural network’s scale through its numbers of nodes and layers. Tuning it balances feature extraction, fitting ability, training time, and generalization. section 5.1.1 section 1.5.2

Tune network size by considering:

Increase layers to extract more features, while changing nodes or model width to adjust the fit. section 5.1.1
Use larger networks when the source data has greater variability or requires more complex solutions. section 5.1.1
Reduce oversized networks to limit memorization and overfitting; undersized networks may underfit varied data. section 1.5.2
Compare size changes experimentally, including their effect on evolutionary convergence. section 6.2.1

Changing the middle layer

A hyperparameter experiment sets middle_layer = 25, then reruns training while changing the layer size. Compare plots and training outcomes to seek the smallest model that trains quickly and performs well. section 5.1.1 section 6.2.1

Listing 5.1 EDL_5_1_Hyperparameter_Tuning.ipynb: Hyperparameters

hp_test = "test 1" #@param {type:"string"}    #1
learning_rate = 3.5e-03
epochs = 500
middle_layer = 25
 
batch_size = 2                                #2
 
data_step = 1                                 #3

Inspect the hyperparameter listing to locate the middle-layer size alongside learning rate, epochs, batch size, and data-step settings. section 5.1.1

How Evolutionary Deep Learning: Genetic algorithms and neural networks covers it

Cover of Evolutionary Deep Learning: Genetic algorithms and neural networks

liveBook manning.com

Read Evolutionary Deep Learning: Genetic algorithms and neural networks on liveBook Get the book at manning.com

Related concepts

parametermeasured by · A parameter is a value or variable that specifies or controls a characteristic, computation, or behavior of a function, model, operation, or request.
learning rateconfigures · A learning rate is a step-size parameter that controls the magnitude of updates made by an iterative learning or optimization process.
genetic algorithmconfigured by · A genetic algorithm is an evolutionary optimization method that improves a population of candidate solutions through fitness-based selection and variation, often including recombination and mutation.
training timeconfigures · Training time is the elapsed computational duration required to fit or optimize a machine-learning model using training data.

This explainer was generated by Manning's knowledge graph from the text of Evolutionary Deep Learning: Genetic algorithms and neural networks. The section links point to the parts of the book it draws on.