liveBook · Manning

Tl.program_id

As explained in GPU Programming with Triton

A program ID is an integer identifying a Triton program instance along a selected dimension of the launch grid.

tl.program_id(axis) returns an integer identifying a Triton program instance’s position along a selected dimension of the launch grid. For vector addition, each instance uses its ID to choose a different chunk of the inputs while running the same kernel code. section 2.2.3 section 2.2.4

A grid can have up to three dimensions. tl.program_id(0) reads the position along the first axis; a two-dimensional kernel can also read axis 1. An ID is unique along its axis, not necessarily across the entire multidimensional grid. section 2.2.3 section 5.3

Mapping a vector to blocks

With pid = tl.program_id(0) and BLOCK_SIZE = 128, compute block_start = pid * BLOCK_SIZE and offsets = block_start + tl.arange(0, BLOCK_SIZE). Program 2 starts at index 256. A mask identifies offsets below the vector length so loads and stores skip invalid positions in the final block. section 2.2.4 section 2.2.9

block_start = pid * BLOCK_SIZE
offsets = block_start + tl.arange(0, BLOCK_SIZE)

The two expressions turn a program ID into a block of vector offsets. section 2.2.4

The host chooses the grid size. For a vector of n_elements, a one-dimensional grid can use triton.cdiv(n_elements, BLOCK_SIZE) entries. Each entry launches one program instance; changing the grid lets the same kernel cover different input sizes. A larger block gives each program more elements and reduces the number of instances, while a smaller block does the reverse. section 2.2.8 section 2.2.4

For a matrix, separate IDs can select row and column tiles, with offsets combined into a two-dimensional pointer array and masks protecting partial tiles. In a row-based rotary-position kernel, a single ID selects a sequence row; input and angle tables require different row strides. If the grid can exceed the row count, guard the row ID before loading memory. section 5.3 section 3.5.3

How GPU Programming with Triton covers it

How deep it goes
Works through ID-based offsets, launch sizing, and bounds checks while deferring compiler implementation details.
Examples
Uses vector addition, matrix addition, and rotary position embedding to illustrate chunk, tile, and row selection.
What it assumes
Assumes intermediate Python and a basic understanding of GPU execution and memory organization.
What you can do after
Apply it via Triton to map program instances to data and guard out-of-range accesses.
How it connects
Connects launch grids, SPMD, vectorized offsets, masking, and memory loads and stores.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.