The policy gradient theorem shows how to improve a parameterized decision policy without differentiating through unknown environment dynamics. Instead of estimating action values and choosing an argmax, the agent adjusts the policy directly. section 9.0 section 9.2
Its gradient weights each policy score, ∇θ log πθ(aₜ|sₜ), by the trajectory return: ∇θJ(θ) = E[Σₜ ∇θ log πθ(aₜ|sₜ) R(τ)]. Positive returns reinforce actions; negative returns suppress them. section 9.8
Cooling control
For data-center cooling, fan speeds create continuous control choices. The theorem supports learning a policy for these choices without discretizing the action space or knowing the cooling system’s transition dynamics. section 9.0 section 9.8
Figure 9.3 Mathematical proof of Policy Gradient Theorem.
Trace how the trajectory-probability factorization and log-derivative steps isolate the policy-dependent terms. section 9.2
