Sampling is useful in variational inference, but a random draw blocks gradients: automatic differentiation cannot see how changing distribution parameters changes the sampled value. The reparameterization trick rewrites that draw as a differentiable transformation of fixed noise, allowing optimization to continue. section 6.5 section 6.15
For a Normal approximating distribution with mean μ and standard deviation σ, generate a sample with θᵢ = μ + σ εᵢ, where εᵢ comes from the standard Normal distribution 𝒩(0, 1). The randomness stays in fixed noise εᵢ; μ and σ enter through addition and multiplication, which are differentiable. section 6.5
Rerouting the gradient
Imagine a water pipe blocked by the sampling step. A direct random draw stops the gradient’s flow. Reparameterization routes the flow around the block: fixed noise enters a simple transformation, and the resulting sample remains connected to μ and σ. Gradient descent can then maximize the ELBO. section 6.5
Use the equation to trace how fixed standard-Normal noise becomes a sample from the parameterized Normal distribution. section 6.5



