liveBook · Manning

Behavior policy

As explained in Applied Reinforcement Learning

A behavior policy is the policy that selects actions and generates experience for a learning algorithm.

A behavior policy selects the actions an agent takes and generates the experience used for learning. In off-policy learning, it can differ from the target policy—the policy the updates aim to improve. section 6.3 section 6.4.2

Q-learning

An agent can use ε-greedy action selection to explore while Q-learning updates toward a greedy target. The experience comes from the behavior policy; the learning target describes what the agent would do if it chose the best action. section 6.4.2

Figure 6.6 Q-learning Algorithm Flowchart.

Follow the Q-learning flow to see where action selection generates experience and where the update uses the greedy target. section 6.4.2

How Applied Reinforcement Learning covers it

How deep it goes
Explains the policy’s role in off-policy learning and contrasts it with the target policy.
Examples
Uses exploratory, safe, or random behavior and ε-greedy action selection as examples.
What you can do after
Understand how an exploratory policy can generate experience while Q-learning targets greedy action values.
How it connects
Connects behavior policies to off-policy learning, target policies, Q-learning, SARSA, and generalized policy iteration.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.