Policy iteration

As explained in Optimization Algorithms: AI techniques for design, planning, and control problems

Policy iteration is an iterative method that alternates between evaluating a policy and improving it until convergence.

Policy iteration is an iterative reinforcement-learning method that alternates between evaluating a policy and improving it until convergence. It seeks an optimal policy that maximizes expected cumulative discounted reward, alongside value iteration and policy gradients. section 12.1.2

The method connects policy choices with value functions: the state-value function estimates expected future reward from a state under a policy, while the discount factor controls the importance of later rewards. section 12.1.2

Figure 12.4 Reach the Treasure game

Use the Reach the Treasure game as a compact reinforcement-learning setting for relating policies, trajectories, and expected returns. section 12.1.2

How Optimization Algorithms: AI techniques for design, planning, and control problems covers it

How deep it goes

Defines its goal and places it alongside value iteration and policy gradients, but does not detail evaluation or improvement steps.

Examples

Uses the Reach the Treasure game and a trajectory with an expected return of 5.728 to illustrate policy-based decision making.

What it assumes

Builds from policies, states, rewards, trajectories, expected returns, value functions, and discount factors.

What you can do after

Evaluate/understand policy iteration as one route toward learning an optimal policy in reinforcement learning.

How it connects

Connects policy iteration with reinforcement learning, Markov decision processes, value iteration, policy gradients, policies, and value functions.

liveBookmanning.com

Read Optimization Algorithms: AI techniques for design, planning, and control problems on liveBook Get the book at manning.com

Related concepts

• dynamic programminguses · Dynamic programming is an algorithmic technique that solves problems by decomposing them into overlapping subproblems and storing or reusing their solutions.
• reinforcement learningkind of · Reinforcement learning is a machine-learning paradigm in which an agent learns actions or a policy through interaction with an environment and evaluative feedback, such as rewards.

This explainer was generated by Manning's knowledge graph from the text of Optimization Algorithms: AI techniques for design, planning, and control problems. The section links point to the parts of the book it draws on.