liveBook · Manning

Policy head

As explained in Applied Reinforcement Learning

A policy head is the output component of a policy model that converts an internal representation into action decisions or action parameters.

A policy head converts a model’s internal representation into action distributions. In a Transformer-based policy, it produces a distribution for each zone; a separate value head estimates the state value used as a baseline. section 9.6

Figure 9.8 Transformer architecture.

The architecture diagram shows where the policy head and value head sit after the Transformer layers. section 9.6

How Applied Reinforcement Learning covers it

How deep it goes
Identifies the per-zone action-distribution output but does not detail the head’s internal computation.
Examples
Uses data-center zones as the setting, producing an action distribution for each zone.
What you can do after
Understand how a Transformer-based policy produces per-zone action distributions.
How it connects
Connects Transformer representations to action distributions and pairs with a value head for state-value estimation.

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.