Stochastic policy

As explained in Optimization Algorithms: AI techniques for design, planning, and control problems

A stochastic policy assigns probabilities to selecting available actions given a state or observation.

A stochastic policy assigns probabilities to selecting available actions given a state or observation. Unlike a deterministic policy, it can choose different actions in the same situation. The policy may be parameterized, with weights and biases adjusted to change behavior through optimization. section 12.1.2

A stochastic policy supports two useful views:

• As a decision rule, it maps states or observations to action probabilities. section 12.1.2
• As a parameterized function, it uses adjustable parameters such as neural-network weights and biases. section 12.1.2

Reach the Treasure

In a four-state game, an agent starts in state 2 and chooses left or right. Moving toward the treasure can produce +10, falling into a fire pit produces −10, and exiting successfully produces +4. A stochastic policy represents the probabilities of choosing each move from the current state. section 12.1.2

Figure 12.4 Reach the Treasure game

Use the Reach the Treasure diagram to trace states, actions, terminal outcomes, and the policy’s decision points. section 12.1.2

How Optimization Algorithms: AI techniques for design, planning, and control problems covers it

How deep it goes

Defines action probabilities and parameterization, but stops short of detailed implementation or optimization procedures.

Examples

Uses the four-state Reach the Treasure game with left-right actions, treasure rewards, fire-pit penalties, and an exit reward.

What you can do after

Apply it via an optimization algorithm to adjust a parameterized agent’s behavior.

How it connects

Connects policies with deterministic policies, MDPs, POMDPs, trajectories, expected return, and optimization algorithms.

liveBookmanning.com

Read Optimization Algorithms: AI techniques for design, planning, and control problems on liveBook Get the book at manning.com

This explainer was generated by Manning's knowledge graph from the text of Optimization Algorithms: AI techniques for design, planning, and control problems. The section links point to the parts of the book it draws on.