Dropout

As explained in Grokking Machine Learning

A neural-network regularization technique that randomly omits units during training to reduce co-adaptation and overfitting.

How Grokking Machine Learning covers it

How deep it goes

Defines random node removal and demonstrates Keras placement, but does not derive dropout mathematically or discuss its inference behavior.

Examples

Uses a circular two-feature classifier and an MNIST digit classifier with dropout layers between dense hidden layers.

What it assumes

Assumes familiarity with neural-network layers and training while explaining dropout within broader discussions of overfitting and backpropagation.

What you can do after

Apply it via Keras by inserting Dropout layers with a chosen probability between neural-network layers.

How it connects

Connects dropout with neural networks, overfitting, regularization, backpropagation, and activation functions.

liveBookmanning.com

Read Grokking Machine Learning on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Deep Learning with R, Third EditionDropout is a regularization technique that randomly disables units during training to reduce reliance on particular activations.
• 

  liveBookmanning.com
  Sutskever's ListDropout is a regularization technique that randomly suppresses units during training to reduce reliance on particular network pathways.
• 

  liveBookmanning.com
  Deep Learning with Python, Third EditionA neural-network regularization technique that randomly zeroes activation entries during training and behaves differently at inference time.
• 

  liveBookmanning.com
  Math and Architectures of Deep LearningA regularization method that randomly turns off units during training to reduce co-adaptation and improve generalization.
• 

  liveBookmanning.com
  Deep Learning with PyTorch, Second EditionA regularization technique that randomly omits activations during training and is turned off or altered for inference.
• 

  liveBookmanning.com
  Evolutionary Deep Learning: Genetic algorithms and neural networksDropout is a neural-network regularization technique that randomly deactivates a proportion of nodes during training.
• 

  liveBookmanning.com
  Natural Language Processing in Action, Second EditionA dropout layer is a neural-network layer that randomly disables units during training to reduce reliance on particular internal representations.
• 

  liveBookmanning.com
  Inside Deep Learning: Math, Algorithms, ModelsA regularization technique that randomly zeros activations or inputs during training to reduce reliance on any single feature or unit.
• 

  liveBookmanning.com
  Machine Learning Algorithms in DepthDropout is a regularization technique that randomly removes or zeroes neural-network connections or activations during training.
• 

  liveBookmanning.com
  LLMs in ProductionDropout is a regularization technique that randomly disables neural-network units during training to reduce co-adaptation and overfitting.
• 

  liveBookmanning.com
  Distributed Machine Learning PatternsA neural-network layer that randomly omits units during training to reduce overfitting and model complexity.

Compare how 12 books explain dropout →

Related concepts

• overfittingaddresses · Overfitting is a modeling condition in which a model fits patterns or idiosyncrasies in its training data too closely and consequently generalizes poorly to new data.
• regularizationkind of · Regularization is a modeling technique that constrains or penalizes solutions during fitting or optimization to discourage undesirable complexity or behavior and improve generalization or estimation.
• AlexNetused by · AlexNet is a convolutional neural network architecture designed for image classification and recognized for its landmark performance in large-scale image recognition.
• evaluation modeconfigured by · Evaluation mode is a model-execution state that changes or disables training-specific behaviors for evaluation or inference.
• hidden layeruses · A hidden layer is an internal neural-network layer between the input and output layers that transforms intermediate representations.

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning. The section links point to the parts of the book it draws on.