liveBook · Manning

Target policy

As explained in Applied Reinforcement Learning

A target policy is the decision rule whose value or action preferences an off-policy learning algorithm estimates or optimizes.

A target policy is the decision rule an off-policy method learns about or aims to optimize. It can differ from the behavior policy that generates experience. section 6.3 section 6.7

Exploring while learning a greedy policy

An agent can use an exploratory behavior policy to sample broadly while updates aim at a greedy target policy. The agent gathers experience one way but learns about another. section 6.3

On-policy learning evaluates and improves the policy used to act. Off-policy learning separates those roles; Q-learning is off-policy, while SARSA is on-policy. section 6.3 section 6.7

How Applied Reinforcement Learning covers it

How deep it goes
Defines the target policy through its contrast with behavior, using exploratory and greedy policies as an illustration.
Examples
Contrasts exploratory behavior that samples broadly with a greedy target policy that the updates aim to optimize.
What you can do after
Understand how an agent can learn about one policy while using another to generate experience.
How it connects
Connects target policies to off-policy learning, behavior policies, generalized policy iteration, Q-learning, and SARSA.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.