liveBook · Manning

Policy

As explained in Timeless Algorithms, The Foundational Ideas

A policy is a rule that specifies how decisions are selected as a process moves through stages or states.

An optimal policy selects an action in each state to maximize expected cumulative reward. The Bellman equation links that choice to immediate reward and discounted expected value across possible future states; the action with the greatest combined value is selected. section 6.5.2

V*(s)=maxa⁡{(R(s, a)+ γ∑s'P(s'| s, a) V*(s')})

The maximization compares available actions using immediate reward and the probability-weighted values of their possible next states. section 6.5.2

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Defines optimal policy through state values, immediate rewards, discounted future values, and action maximization.
What you can do after
Understand how state values determine an action that maximizes expected cumulative reward.
How it connects
Connects optimal policy to the Bellman equation, dynamic programming, value iteration, and value functions.

Also explained in

Compare how 21 books explain policy →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.