liveBook · Manning

Block size

As explained in GPU Programming with Triton

Block size specifies the number of elements or threads assigned to a GPU computation block or Triton program instance.

Block size sets how many elements a Triton program instance handles in a chunk. For a vector kernel, program ID `pid` selects the chunk start (`pid * BLOCK_SIZE`), and local indices become global offsets when added to that start. section 2.2.4 section 2.2.5

The caller chooses `BLOCK_SIZE`. A larger value gives each program more work and launches fewer programs; a smaller value does the reverse. For NVIDIA hardware, a multiple of 32 is a useful starting point, and 128 is used for vector addition—not a universal optimum. section 2.2.4

Marking `BLOCK_SIZE` as `tl.constexpr` makes it a compile-time constant. Triton specializes the generated code for that value, including loads, address calculations, and stores. This differs from a runtime argument such as the vector length, which can vary without creating a separate compiled variant for each length. section 2.2.2

block_start = pid * BLOCK_SIZE
offsets = block_start + tl.arange(0, BLOCK_SIZE)

See how the program ID and block size determine a chunk’s start and element offsets. section 2.2.4

Vector addition

With `BLOCK_SIZE = 128`, program ID 2 starts at element 256. In RMSNorm, block lanes also process elements in chunks; masks keep loads and stores within the row length when it is not a multiple of the block size. section 2.2.4 section 4.1.4

How GPU Programming with Triton covers it

How deep it goes
Defines chunk indexing, compile-time specialization, and launch trade-offs, but leaves performance tuning for later experimentation.
Examples
Uses vector addition with block size 128 and RMSNorm chunks guarded by masks.
What you can do after
Configure Triton kernel chunks and reason about how block size changes program count and per-program work.
How it connects
Connects program IDs and SPMD execution to `tl.arange`, compile-time constants, and NVIDIA warps.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.