An optimal policy selects an action in each state to maximize expected cumulative reward. The Bellman equation links that choice to immediate reward and discounted expected value across possible future states; the action with the greatest combined value is selected. section 6.5.2
The maximization compares available actions using immediate reward and the probability-weighted values of their possible next states. section 6.5.2




















