liveBook · Manning

Offset

As explained in GPU Programming with Triton

An offset is an index displacement used to address an element within a data region.

An offset is a displacement from a tensor’s base pointer, measured in elements. For a logical index (i₀, …, iₙ₋₁), the flat offset is the sum of each index multiplied by its dimension’s stride. section 5.2

In Triton kernels, offsets translate parallel work coordinates into memory addresses:

  • Combine a program’s ID-derived starting position with local indices to address a block of elements; use masks to protect out-of-range positions. section 2.4
  • For multidimensional data, calculate coordinates along each axis and combine them with tensor strides; one flat offset may not describe a matrix tile. section 5.3.1
Listing 3.3 Shared-frequency RoPE kernel (LLaMA-style)
@triton.jit
def rope_kernel(
    x_ptr,
    output_ptr,
    cos_ptr,
    sin_ptr,
    seq_len,
    feature_dim,
    BLOCK_SIZE: tl.constexpr,
):

    pid = tl.program_id(0) #1
    half_dim = feature_dim // 2 #2
    local_idx = tl.arange(0, BLOCK_SIZE)
    mask = local_idx < half_dim
    row_start = pid * feature_dim
    first_offset = row_start + local_idx
    second_offset = row_start + local_idx + half_dim #3

    x1 = tl.load(x_ptr + first_offset, mask=mask, other=0.0)
    x2 = tl.load(x_ptr + second_offset, mask=mask, other=0.0)

    angle_row_start = pid * half_dim #4
    angle_offset = angle_row_start + local_idx

    cos_val = tl.load(cos_ptr + angle_offset, mask=mask, other=0.0) #5
    sin_val = tl.load(sin_ptr + angle_offset, mask=mask, other=0.0)

    out1 = x1 * cos_val - x2 * sin_val #6
    out2 = x2 * cos_val + x1 * sin_val

    tl.store(output_ptr + first_offset, out1, mask=mask)
    tl.store(output_ptr + second_offset, out2, mask=mask)

The RoPE kernel calculates offsets for paired input values and angle values, whose tensors have different strides. section 3.5.3

How GPU Programming with Triton covers it

How deep it goes
Defines flat offsets as index–stride sums and works through multidimensional mappings, while leaving thread-level execution to Triton’s compiler.
Examples
Uses vector and matrix indexing, plus RoPE’s paired input and angle offsets, to illustrate address calculation.
What it assumes
Relies on tensor dimensions, logical indices, and Triton program IDs; defines how strides map indices to flat memory offsets.
What you can do after
Build memory addresses from logical indices and strides, then check bounds and ensure output locations are assigned correctly.
How it connects
Connects offsets to tensor strides, program IDs, masks, and pointer arithmetic in Triton kernels.

Also explained in

Compare how 5 books explain offset →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.