Reward-to-go, written Gₜ, is the discounted sum of rewards from time t onward. In policy-gradient estimates, it credits an action only for later rewards, avoiding noise from rewards received before that action. section 9.3
Gₜ = rₜ + γ rₜ₊₁ + γ² rₜ₊₂ + ... = Σₖ₌ₜᵀ γᵏ⁻ₜ rₖ
The equation expresses reward-to-go as the sum of future rewards, discounted by their distance from time t. section 9.3

