liveBook · Manning

2d rotation

As explained in Build a DeepSeek Model (From Scratch)

A 2D rotation is a geometric transformation that rotates a two-dimensional vector by an angle while preserving its length.

A 2D rotation turns each paired slice of a high-dimensional Query or Key vector through a position-dependent angle, injecting token position while preserving the pairwise geometric structure. section 3.11.2

RoPE applies the operation in three practical steps:

  • Group dimensions into independent pairs: a 4D vector [x1, x2, x3, x4] becomes (x1, x2) and (x3, x4). section 3.11.2
  • Rotate every pair by an angle θ determined by token position; different pairs may use different angles. section 3.11.2
  • Reassemble the rotated pairs into the position-encoded Query or Key vector. section 3.11.2
Figure 3.24 The dimensions of a Query or Key vector are paired up for rotation.

Use the dimension-pairing figure to visualize how high-dimensional Query and Key vectors become independent 2D rotation planes. section 3.11.2

How Build a DeepSeek Model (From Scratch) covers it

How deep it goes
Explains pairing, position-dependent rotation, and reassembly, with 4D and 128D illustrations.
Examples
Uses a 4D vector split into two pairs and generalizes the operation to 64 pairs in a 128-dimensional vector.
What it assumes
Assumes high-dimensional Query and Key vectors while teaching pairwise rotation from a simple 4D example.
What you can do after
Evaluate/understand how RoPE transforms Query and Key vectors through pairwise, position-dependent rotations.
How it connects
Connects 2D rotation to RoPE, Query and Key vectors, positional encoding, and sinusoidal encoding.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build a DeepSeek Model (From Scratch). The section links point to the parts of the book it draws on.