liveBook · Manning

Reparameterization trick

As explained in Applied Reinforcement Learning

A stochastic-gradient technique that expresses a random sample as a deterministic transformation of parameter-independent noise.

The reparameterization trick makes a stochastic policy’s sampling step differentiable. Draw parameter-independent noise ε from a standard normal, then compute an action as μ + σε. Randomness enters through ε, while the deterministic transformation carries gradients to the policy’s mean μ and standard deviation σ. section 9.7

Cooling controls

For each data-center zone, a Gaussian policy samples fan and valve actions using this method. A sigmoid then maps the raw samples into the environment’s required 0–1 range. section 9.7

How Applied Reinforcement Learning covers it

How deep it goes
Explains the noise transformation and its gradient benefit, then shows its use in Gaussian action sampling.
Examples
Illustrates sampling fan and valve controls for data-center cooling, then squashing actions into the 0–1 range.
What you can do after
Apply Normal.rsample() to generate differentiable stochastic actions for policy learning.
How it connects
Connects Gaussian policies, stochastic action sampling, and backpropagation.

Also explained in

Compare how 4 books explain reparameterization trick →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.