liveBook · Manning

Stochastic policy

As explained in Applied Reinforcement Learning

A stochastic policy is a decision rule that assigns a probability distribution over possible actions for each state.

A stochastic policy maps a state to a probability distribution over possible actions, rather than always selecting one fixed action. The agent samples an action from that distribution. In continuous control, the policy can output distribution parameters such as a Gaussian’s mean and standard deviation. section 9.1

Randomness can serve practical purposes:

  • A security patrol can vary its route, making its behavior harder for an intruder to exploit. section 9.1
  • A fan controller can express uncertainty across possible speeds; as it learns, its distribution can narrow, making exploration part of the policy rather than an external schedule. section 9.1
  • A language model treats the current context as the basis for a distribution over next tokens. It can choose the most probable token or sample for more varied responses. section 10.2

Policy-gradient methods learn stochastic policies directly. In language-model reinforcement learning, generated responses receive scores from a reward model, and the policy is updated to make safer, more helpful, and more empathetic responses more probable while preserving useful language abilities. section 9.1 section 10.2

Figure 9.1 Value-Based vs. Policy-Based Reinforcement Learning.

Compare value-based and policy-based approaches to see where stochastic policies fit. section 9.1

How Applied Reinforcement Learning covers it

How deep it goes
Defines action distributions, illustrates Gaussian parameters and sampling, and explains policy-driven exploration without deriving update equations.
Examples
Uses a security patrol varying routes, a fan choosing speeds, and a language model generating tokens.
What you can do after
Understand how sampling supports exploration and varied token generation, and why randomized patrol behavior can be harder to exploit.
How it connects
Connects policy gradients, value-based methods, continuous control, exploration, Markov decision processes, and reward models.

Also explained in

Compare how 2 books explain stochastic policy →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.