Batch size is the number of input examples processed together during one forward pass or training update. It is a practical training hyperparameter: changing it can affect how long training takes and how well a model fits the target. section 5.1.1
Treat batch size as part of a linked configuration rather than an isolated number. When data_step generates substantially more training data, increasing batch_size can prevent the training time for a fixed number of epochs from growing dramatically. The useful value depends on the dataset and model, so compare runs instead of assuming one setting always works. section 5.1.1 section 9.1.3
Compare batch-size settings
In a notebook experiment, set batch_size to 2, run the initial 500-epoch training, then change the amount of generated data by lowering data_step. Repeat the run with a larger batch size and compare training time and plotted results. In a GAN exercise, increase or decrease BATCH_SIZE and inspect how training changes. section 5.1.1 section 9.1.3
Listing 5.1 EDL_5_1_Hyperparameter_Tuning.ipynb: Hyperparameters
hp_test = "test 1" #@param {type:"string"} #1
learning_rate = 3.5e-03
epochs = 500
middle_layer = 25
batch_size = 2 #2
data_step = 1 #3
Inspect the hyperparameter listing showing batch_size alongside learning rate, epochs, model width, and data_step. section 5.1.1
