liveBook · Manning

Positional embedding

As explained in Applied Reinforcement Learning

A representation added to inputs to encode their positions or identities so attention models can use ordering or spatial relationships.

In a Transformer processing cooling zones, sinusoidal signals encode each zone’s position: zone 1 receives a different encoding than zone 5. This helps the model learn spatial relationships alongside similarities in zone state. section 9.6 section 9.8

Figure 9.8 Transformer architecture.

The architecture diagram shows where positional encoding enters the Transformer. section 9.6

How Applied Reinforcement Learning covers it

How deep it goes
Describes sinusoidal functions that distinguish zones and support spatial learning, without specifying their formula.
Examples
Uses data-center cooling zones, contrasting zone 1 with zone 5 and describing zone adjacency.
What you can do after
Understand how sinusoidal zone signals let a Transformer learn spatial relationships alongside state similarity.
How it connects
Connects positional encoding with Transformers, multi-head self-attention, and spatial relationships among zones.

Also explained in

Compare how 20 books explain positional embedding →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.