Entropy measures how spread out a probability distribution is: high entropy means greater uncertainty, while low entropy means probability is concentrated on one action. In policy-gradient training, entropy regularization rewards policies for staying uncertain enough to explore, reducing the risk of committing to a strategy too early. The objective adds a bonus, βH(πθ), to expected return; β is typically 0.01. section 9.4
A Gaussian action policy
For a policy with mean 0.5, σ = 0.3 gives a broad distribution over fan speeds and high entropy; σ = 0.01 concentrates actions near 0.5 and has low entropy. Entropy often falls as a policy gains confidence, but falling too quickly can prevent it from exploring better alternatives. section 9.4


















