A program instance is Triton’s unit of work: one execution of a kernel at a position in the launch grid. A kernel is written from the perspective of one instance operating on a block, or tile, of data. Many instances run the same compiled logic on different data. Triton manages the individual threads within the block; the corresponding CUDA term is block. section 1.3.3 section 2.1.1 section 2.2
Vector addition
For two vectors of eight elements with a block size of four, two instances do the work. The first adds elements 0–3; the second adds elements 4–7. Each performs a vectorized block-level operation, not necessarily a single hardware instruction. section 2.1.1
Figure 2.1 CUDA assigns one thread to each element, so two blocks of four threads (T0-T3) each compute four scalar additions individually. Triton assigns one program instance to each whole block, so program P0 computes z[0:4] = x[0:4] + y[0:4] and program P1 computes z[4:8] = x[4:8] + y[4:8] as single block-level operations.
Compare the two Triton instances with the individual CUDA threads assigned to the same eight additions. section 2.1.1
For a one-dimensional vector, the host can launch ceil(n_elements / BLOCK_SIZE) instances using triton.cdiv. Each instance gets its position with tl.program_id(0), computes block_start = pid * BLOCK_SIZE, and adds tl.arange(0, BLOCK_SIZE) to form element offsets. A mask checks offsets against n_elements so loads and stores skip positions beyond the end of the vector. section 2.2.3 section 2.2.8 section 2.2.9
A launch grid can have up to three dimensions; the axis passed to tl.program_id selects the coordinate to retrieve. The GPU schedules instances across streaming multiprocessors as resources become available. The compiler handles thread-level execution within each instance. section 2.2.3 section 2.4
The mapping from instances to data depends on the kernel. For example, a rotary-position-embedding kernel can assign one sequence row to each instance. If its grid has more entries than rows, it needs a row-bound guard before loading; a mask for feature positions alone does not protect an invalid row. For a reduction larger than one instance can cover, partial results can instead be reduced through successive launches. section 3.5.3 section 4.2.2
