liveBook · Manning

Value iteration

Value iteration is an iterative method that applies Bellman updates to state values to approximate optimal expected returns and determine an optimal policy.

How 2 Manning books explain it

Applied Reinforcement Learning

Value iteration is a dynamic-programming algorithm that repeatedly applies a Bellman optimality update to approximate optimal state values and derive an optimal policy.

How it is used
Plans decisions in modeled Markov decision processes by refining state values and policies.
How deep it goes
Explains single-sweep evaluation and greedy improvement, then implements the loop and tracks convergence.
Examples
Works through project-budget allocation in code and a simplified weekly inventory-ordering problem.
What you can do after
Builds it from scratch for a modeled resource-allocation problem and interprets the resulting policy.

Read how Applied Reinforcement Learning explains value iteration →

Optimization Algorithms: AI techniques for design, planning, and control problems

Value iteration is an iterative reinforcement-learning or dynamic-programming method that updates state values toward optimal expected returns.

How it is used
Positions value iteration as an iterative route toward an optimal reinforcement-learning policy.
How deep it goes
Defines its reinforcement-learning role and related value functions but does not provide update steps or implementation details.
Examples
Uses the Reach the Treasure game, an agent-environment figure, and a worked expected-return calculation yielding 5.728.
What you can do after
Evaluate how state values, action values, policies, and discounting support maximizing expected cumulative reward.

Read how Optimization Algorithms: AI techniques for design, planning, and control problems explains value iteration →

Related concepts

Questions about value iteration

What is value iteration?

Value iteration is an iterative method that applies Bellman updates to state values to approximate optimal expected returns and determine an optimal policy.

Which Manning books explain value iteration?

Applied Reinforcement Learning and Optimization Algorithms: AI techniques for design, planning, and control problems.

What is value iteration a kind of?

Value iteration is a kind of dynamic programming. Dynamic programming is an algorithmic technique that solves problems by decomposing them into overlapping subproblems and storing or reusing their solutions.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.