Dropout
Dropout is a neural-network regularization technique that randomly suppresses selected units or activations during training to reduce overfitting.
Grokking Machine Learning
A neural-network regularization technique that randomly omits units during training to reduce co-adaptation and overfitting.
- How it is used
- Reduces neural-network overfitting during training by preventing a few units from dominating weight updates.
- How deep it goes
- Defines random node removal and demonstrates Keras placement, but does not derive dropout mathematically or discuss its inference behavior.
- Examples
- Uses a circular two-feature classifier and an MNIST digit classifier with dropout layers between dense hidden layers.
- What you can do after
- Apply it via Keras by inserting Dropout layers with a chosen probability between neural-network layers.
Build AI Drug Discovery Pipelines
Dropout is a regularization technique that randomly disables units during training to reduce reliance on particular representations.
- How it is used
- Uses dropout to limit overfitting in drug-discovery neural networks, including an active-learning surrogate.
- How deep it goes
- Explains random zeroing across training batches and its removal during inference, with layer-level implementations.
- Examples
- Uses a kinase-binder network and a docking-score surrogate with dropout probabilities of 0.2 and 0.5, respectively.
- What you can do after
- Apply dropout via PyTorch nn.Dropout in hidden layers and switch to evaluation mode for prediction.
Read how Build AI Drug Discovery Pipelines explains dropout →
Deep Learning with R, Third Edition
Dropout is a regularization technique that randomly disables units during training to reduce reliance on particular activations.
- How it is used
- Uses dropout to improve generalization after a model can overfit, including in image classifiers and stacked Transformer decoders.
- How deep it goes
- Explains random zeroing, dropout rates and training-versus-test scaling, then demonstrates layer placement in complete models.
- Examples
- Demonstrates dropout in IMDb review classification, an augmented image classifier and a stacked Transformer decoder.
- What you can do after
- Apply it via R Keras dropout layers and compare model configurations using validation data.
Read how Deep Learning with R, Third Edition explains dropout →
Sutskever's List
Dropout is a regularization technique that randomly suppresses units during training to reduce reliance on particular network pathways.
- How it is used
- Reduces overfitting in AlexNet’s parameter-heavy fully connected layers and motivates later regularization of LSTMs.
- How deep it goes
- Explains random neuron suppression, training-versus-test scaling, and an ensemble analogy; leaves selective LSTM dropout at a high level.
- Examples
- Uses AlexNet’s 50% dropout in FC6 and FC7, a network comparison figure, and selective LSTM dropout.
- What you can do after
- Evaluate/understand it as an overfitting countermeasure, distinguishing training-time suppression from test-time behavior.
Deep Learning with Python, Third Edition
A neural-network regularization technique that randomly zeroes activation entries during training and behaves differently at inference time.
- How it is used
- Uses dropout to improve generalization after a model begins overfitting.
- How deep it goes
- Explains random masking and training–inference scaling, then details temporally constant masks for recurrent networks.
- Examples
- Uses an activation vector, an IMDb review classifier, and a temperature-forecasting LSTM.
- What you can do after
- Apply it via Keras Dropout layers and recurrent-layer dropout arguments.
Read how Deep Learning with Python, Third Edition explains dropout →
Grokking Machine Learning, Second Edition
A neural-network layer that randomly omits units during training to reduce reliance on specific pathways.
- How it is used
- Reduces neural-network overfitting by preventing a few strong nodes or pathways from dominating training.
- How deep it goes
- Explains the intuition, random node-removal process, probability hyperparameter, and Keras placement, but not deeper mathematical internals.
- Examples
- Uses a two-arm gym analogy, a circular two-class dataset, Keras layers, and a two-hidden-layer architecture with probability 0.2.
- What you can do after
- Apply dropout via Keras by inserting Dropout layers and tuning their probability with other neural-network hyperparameters.
Read how Grokking Machine Learning, Second Edition explains dropout →
Math and Architectures of Deep Learning
A regularization method that randomly turns off units during training to reduce co-adaptation and improve generalization.
- How it is used
- Regularizes neural-network training by reducing overfitting and limiting reliance on a small number of units.
- How deep it goes
- Develops the definition, Bernoulli-mask formulation, co-adaptation rationale, subnetwork interpretation, and PyTorch implementation.
- Examples
- Uses an eight-subnetwork three-node example and a PyTorch sequential model containing two dropout layers.
- What you can do after
- Apply dropout via PyTorch by inserting torch.nn.Dropout layers into a neural-network architecture.
Read how Math and Architectures of Deep Learning explains dropout →
Grokking Bayes
Dropout is a neural-network regularization technique that randomly omits units or connections during training to reduce co-adaptation and overfitting.
- How it is used
- Regularizes neural networks during training and estimates predictive uncertainty through Monte Carlo dropout.
- How deep it goes
- Explains dropped units, Bernoulli architecture samples, stochastic prediction, and PyTorch implementation without deriving the full variational objective.
- Examples
- Uses dropped-node diagrams, a PyTorch dropout layer, and 2,000 repeated forward passes to generate predictive samples.
- What you can do after
- Operate/configure it by adding PyTorch dropout and repeated prediction passes, then compute predictive means and standard deviations.
Deep Learning with PyTorch, Second Edition
A regularization technique that randomly omits activations during training and is turned off or altered for inference.
- How it is used
- Applies dropout as regularization to reduce overfitting in convolutional neural networks.
- How deep it goes
- Defines random activation omission, explains its anti-memorization rationale, and demonstrates PyTorch placement without developing mathematical internals.
- Examples
- Demonstrates a two-convolution NetDropout model using Dropout2d with probability 0.4 after activation and pooling.
- What you can do after
- Apply it via PyTorch by inserting the appropriate dropout module and switching the model between training and evaluation modes.
Read how Deep Learning with PyTorch, Second Edition explains dropout →
Evolutionary Deep Learning: Genetic algorithms and neural networks
Dropout is a neural-network regularization technique that randomly deactivates a proportion of nodes during training.
- How it is used
- Uses random node deactivation to reduce neural-network memorization and improve feature generalization.
- How deep it goes
- Explains iteration-level behavior, placement, parameterization, and overuse, but does not derive dropout mathematically.
- Examples
- Demonstrates Dropout(0.5) in convolutional autoencoder encoder and decoder layers, alongside batch normalization.
- What you can do after
- Apply it via Dropout layers, adjust the disabled-neuron proportion, and evaluate whether the architecture improves validation behavior.
Read how Evolutionary Deep Learning: Genetic algorithms and neural networks explains dropout →
Natural Language Processing in Action, Second Edition
A dropout layer is a neural-network layer that randomly disables units during training to reduce reliance on particular internal representations.
- How it is used
- Improves NLP neural-network robustness by reducing overfitting and dependence on individual learned representations.
- How deep it goes
- Explains training intuition, CNN placement, ratio selection, and tuning effects, but does not detail implementation internals.
- Examples
- Uses text CNNs, RNN language models, tweet classification, learning curves, and recorded hyperparameter experiments.
- What you can do after
- Apply dropout via a neural-network layer and tune its percentage to control overfitting in NLP models.
Read how Natural Language Processing in Action, Second Edition explains dropout →
Inside Deep Learning: Math, Algorithms, Models
A regularization technique that randomly zeros activations or inputs during training to reduce reliance on any single feature or unit.
- How it is used
- Uses dropout to regularize networks and corrupt inputs for denoising autoencoders.
- How deep it goes
- Defines random feature zeroing and demonstrates layer placement in PyTorch, without detailing dropout internals.
- Examples
- Demonstrates input and hidden-layer dropout in an autoencoder and tests reconstruction with Gaussian and dropout noise.
- What you can do after
- Apply dropout in PyTorch by setting an input probability and using default dropout in hidden layers.
Read how Inside Deep Learning: Math, Algorithms, Models explains dropout →
Machine Learning Algorithms in Depth
Dropout is a regularization technique that randomly removes or zeroes neural-network connections or activations during training.
- How it is used
- Regularizes multilayer perceptrons to reduce overfitting and improve generalization.
- How deep it goes
- Defines random connection removal and contrasts dropout with weight decay and early stopping, but gives no mathematical derivation.
- Examples
- Uses a dropout diagram and a Keras/TensorFlow multilayer perceptron trained on normalized MNIST images.
- What you can do after
- Apply it via Keras and TensorFlow when regularizing a multilayer perceptron.
Read how Machine Learning Algorithms in Depth explains dropout →
LLMs in Production
Dropout is a regularization technique that randomly disables neural-network units during training to reduce co-adaptation and overfitting.
- How it is used
- Regularizes recurrent neural networks during training to reduce overfitting and co-adaptation.
- How deep it goes
- Explains random unit disabling and co-adaptation prevention, then demonstrates dropout as an LSTM configuration without deriving its internals.
- Examples
- Uses a Twitter sentiment dataset and a PyTorch RNN/LSTM implementation to illustrate dropout during training.
- What you can do after
- Apply dropout through an LSTM configuration and assess its generalization using validation performance.
Timeless Algorithms, The Foundational Ideas
Dropout is a neural-network regularization technique that randomly sets selected unit outputs to zero during training.
- How it is used
- Constrains how a neural network uses capacity to reduce dependence on training-specific observations.
- How deep it goes
- Defines temporary zeroing of randomly selected hidden-node outputs and explains its representational effect, but gives no implementation details.
- What you can do after
- Evaluate dropout as a way to discourage repeated reliance on the same hidden nodes during training.
Read how Timeless Algorithms, The Foundational Ideas explains dropout →
Distributed Machine Learning Patterns
A neural-network layer that randomly omits units during training to reduce overfitting and model complexity.
- How it is used
- Regularizes neural networks during training to reduce overfitting and improve generalization.
- How deep it goes
- Defines the layer’s purpose, placement, and example rates without explaining its internal mathematical mechanism.
- Examples
- Uses MNIST digit classification and a CNN variation that drops half of its hidden units.
- What you can do after
- Apply it via TensorFlow Keras by inserting and configuring a Dropout layer in a sequential model.
Read how Distributed Machine Learning Patterns explains dropout →
Also covered in
- Graph Neural Networks in Action
- Machines that ThinkIn early access (MEAP)
- Build a Large Language Model (From Scratch)
- Fight Fraud with Machine LearningIn early access (MEAP)
- Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
What to know first
- overfittingOverfitting is a modeling condition in which a model adapts too closely to particular data, including its noise or idiosyncrasies, and consequently generalizes poorly to new data.
- objective functionAn objective function is a mathematical function that assigns values to candidate solutions or parameter settings, which an optimization procedure seeks to minimize or maximize.
- regularizationRegularization is a modeling technique that constrains learning or penalizes model complexity to discourage overfitting and improve generalization.
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
Related concepts
- overfittingaddresses · Overfitting is a modeling condition in which a model adapts too closely to particular data, including its noise or idiosyncrasies, and consequently generalizes poorly to new data.
- regularizationkind of · Regularization is a modeling technique that constrains learning or penalizes model complexity to discourage overfitting and improve generalization.
- AlexNetused by · AlexNet is a convolutional neural network architecture designed for image classification and recognized for its landmark performance in large-scale image recognition.
- evaluation modeconfigured by · Evaluation mode is a model or module state in which training-dependent layer behaviors, such as dropout or batch-statistics updates, switch to evaluation or inference behavior.
- Monte Carlo dropoutused by · Monte Carlo dropout is an inference technique that retains stochastic dropout across repeated forward passes to approximate a distribution of model predictions.
Questions about dropout
What is dropout?
Dropout is a neural-network regularization technique that randomly suppresses selected units or activations during training to reduce overfitting.
Which Manning books explain dropout?
Grokking Machine Learning, Build AI Drug Discovery Pipelines, Deep Learning with R, Third Edition, Sutskever's List, Deep Learning with Python, Third Edition, Grokking Machine Learning, Second Edition, Math and Architectures of Deep Learning, Grokking Bayes and 8 more.
What should I know before learning dropout?
Manning books that teach dropout usually introduce overfitting, objective function, regularization and language model first.
What is dropout a kind of?
Dropout is a kind of regularization. Regularization is a modeling technique that constrains learning or penalizes model complexity to discourage overfitting and improve generalization.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.















