Dropout is a neural-network layer that randomly removes hidden units during training. It reduces overfitting, limits model complexity, and can speed computation. section 9.2.1 section 8.1.1
Implement dropout as a model-architecture choice:
- Insert a Dropout layer alongside dense or other neural-network layers. section 8.1.1
- Set a rate such as 0.2, or drop half of the hidden units in a model variation. section 9.2.1 section 8.1.1
- Compare its training behavior and accuracy with a basic model or a batch-normalized variation. section 9.2.1
MNIST classifier
A TensorFlow Keras sequential model flattens 28 × 28 handwritten-digit images, applies a dense ReLU layer, inserts Dropout(0.2), and predicts ten classes with softmax. section 8.1.1
Listing 9.4 Defining the basic CNN model
def build_and_compile_cnn_model():
print("Training CNN model")
model = models.Sequential()
model.add(layers.Input(shape=(28, 28, 1), name='image_bytes'))
model.add(
layers.Conv2D(32, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.Flatten())
model.add(layers.Dense(64, activation='relu'))
model.add(layers.Dense(10, activation='softmax'))
model.summary()
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
return model
Inspect the surrounding TensorFlow Keras CNN definition to see where a dropout variation fits within convolutional, pooling, flattening, and dense layers. section 9.2.1
