RotaryPositionalEncoding generates rotation frequencies and applies them to query and key representations, injecting positional information by rotating pairs of dimensions. It precomputes frequency values and supports position offsets for cached inference, where each step processes a new token. section 6.2.1
The implementation follows a compact precompute-and-apply pipeline:
- Slices rotations using position_offset so cached tokens and newly processed tokens receive aligned positions. section 6.2.1
