liveBook · Manning

Shared-frequency design

As explained in GPU Programming with Triton

A shared-frequency design uses one sine and cosine value for each pair of input elements rather than independent values for every element.

In this RoPE layout, each feature pair, xᵢ and xᵢ₊D⁄₂, uses the same rotation angle. Cosine and sine tables therefore store one value per pair, with shape [sequence length, D/2]. section 3.5.2 section 3.6.3

A Triton kernel must map each sequence row into two different layouts:

  • Load paired input features across the midpoint of a row of width D. section 3.5.2
  • Load the shared cosine and sine values from the corresponding row of width D/2; keeping the row offsets separate avoids mixing the layouts. section 3.5.3
  • Reuse each angle pair for two input elements, reducing auxiliary-table loads compared with a full elementwise design. section 3.5.3 section 3.6.3
Figure 3.9 A tensor with shape [4,8] stored in row-major order. The 2D structure (top) maps to a flat memory layout (bottom) where all elements of each row are stored consecutively. Within each row, element i pairs with element i+4 for the rotation operation. The pairing arrows show how positions 0 and 4, 1 and 5, 2 and 6, 3 and 7 form the four pairs for Position 0.
CH03 F09 Fartale

The diagram traces how a width-eight row pairs positions 0–3 with positions 4–7. section 3.5.2

How GPU Programming with Triton covers it

How deep it goes
Maps paired input offsets to half-width angle tables and contrasts their loads with full elementwise tables.
Examples
Shows width-eight pairs 0–4 through 3–7 and a shared-frequency Triton RoPE kernel.
What you can do after
Apply shared-angle rotations in a Triton RoPE kernel by mapping paired features to half-width angle tables.
How it connects
Connects rotary position embedding with Triton kernel coordinate mapping.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.