A stochastic policy maps a state to a probability distribution over possible actions, rather than always selecting one fixed action. The agent samples an action from that distribution. In continuous control, the policy can output distribution parameters such as a Gaussian’s mean and standard deviation. section 9.1
Randomness can serve practical purposes:
- A security patrol can vary its route, making its behavior harder for an intruder to exploit. section 9.1
- A fan controller can express uncertainty across possible speeds; as it learns, its distribution can narrow, making exploration part of the policy rather than an external schedule. section 9.1
- A language model treats the current context as the basis for a distribution over next tokens. It can choose the most probable token or sample for more varied responses. section 10.2
Policy-gradient methods learn stochastic policies directly. In language-model reinforcement learning, generated responses receive scores from a reward model, and the policy is updated to make safer, more helpful, and more empathetic responses more probable while preserving useful language abilities. section 9.1 section 10.2
Figure 9.1 Value-Based vs. Policy-Based Reinforcement Learning.
Compare value-based and policy-based approaches to see where stochastic policies fit. section 9.1

