concept variance in category deep learning

appears as: varice, varice, variance, The varice
Deep Reinforcement Learning in Action

This is an excerpt from Manning's book Deep Reinforcement Learning in Action.

The most common example of a PDF is the normal (also known as Gaussian, or just bell-curve) distribution. If we have a probability with a continuous action, say a car game where we need to control the velocity of the car from 0 to some maximum value, which is a continuous variable, how might we do this with a policy network? Well, we could drop the idea of probability distribution and just train the network to produce the single value of velocity that it predicts is best, but then we’d risk not exploring enough (and it is difficult to train such a network). A lot of power comes from a little bit of randomness. The kind of neural networks we employ in this book only produce vectors (or tensors more generally) as output, so they can’t produce a continuous probability distribution—we have to be more clever. A PDF like a normal distribution is defined by two parameters, the mean and variance. Once we have those, we have a normal distribution that we can sample from. So we can just train a neural network to produce mean and standard deviation values that we can then plug into the normal distribution equation and sample from that.

Figure 5.4. The bias-variance tradeoff is a fundamental machine learning concept that says any machine learning model will have some degree of systematic deviation from the true data distribution and some degree of variance. You can try to reduce the variance of your model, but it will always come at the cost of increased bias.

Another question we can ask of a probability distribution is its spread or variance. Our beliefs about something can be more or less confident, so a probability distribution can be narrow or wide respectively. The calculation of variance uses the expectation operator and is defined as ], but don’t worry about remembering this equation—we will use built-in numpy functions to compute variance. Variance is either denoted Var(X) or σ2 (sigma squared) where is the standard deviation, so the variance is the standard deviation squared. The μ in this equation is the standard symbol for mean, which again is , where X is a random variable of interest.