Rotation changes a vector’s orientation while preserving its length. In sinusoidal positional encoding, each even–odd dimension pair forms a 2D vector on a unit circle. Moving from position p to p + k rotates that vector by an angle determined by k and the pair’s frequency, turning relative position into a simple geometric relationship. section 3.10.2
The rotation angle begins with the position-dependent frequency θ = p / 10000^(2i / d_model); the same frequency determines how far a dimension pair rotates as position changes. section 3.10.2
Rotation inside attention
Rotary Positional Encoding applies rotation directly to Query and Key vectors inside attention instead of adding positional vectors to token embeddings. This injects position where attention uses it, while preserving the learned magnitude of each vector; only its orientation changes. section 3.11.1




