A neural network can give different answers for the same uncertain case. Monte Carlo dropout estimates that uncertainty by keeping dropout active during prediction and repeating the forward pass. Each pass samples a slightly different network, producing a collection of predictions. Their average gives a mean estimate, while their spread estimates predictive uncertainty. section 10.6 section 10.9
Apply Monte Carlo dropout in a few practical steps:
- Train the dropout network with ordinary gradient-based optimization. section 10.7
- Leave dropout enabled when making predictions; in PyTorch, avoid calling model.eval(). section 10.7
- Run many forward passes for each test input and collect the resulting predictions. section 10.7
- Compute the sample mean and standard deviation to summarize the prediction and its uncertainty. section 10.6 section 10.7
A PyTorch prediction loop
A model can define an ordinary dropout layer, then sample predictions repeatedly: `T = 2000; predictions = torch.stack([model(x_test) for _ in range(T)])`. The resulting tensor contains approximate predictive samples from which the mean and standard deviation can be calculated. section 10.7
Figure 10.12 At test time, for a given input, we perform multiple forward passes with dropout enabled to obtain a distribution of predictions, which serves as our predictive distribution for that input.
Visualizes how multiple dropout-enabled forward passes turn one input into a distribution of predictions. section 10.6

