Block-level programming expresses a GPU kernel from the perspective of one program instance processing a tile of data. A launch grid creates instances for the full workload, and each instance applies the same kernel logic to its own slice. This is also called blocked or tiled SPMD (single program, multiple data). section 2.1.1 section 2.1.2
In Triton, the programmer chooses the tile size and writes vectorized loads, computation, stores, and masks for the tile. The compiler and runtime handle the distribution of work among threads within a block. By contrast, CUDA code is written from a thread’s perspective and requires explicit thread indexing and coordination. section 1.5 section 2.1.1
Vector addition in two tiles
For vectors x and y of length 8, choose a block size of 4 and launch two program instances. The first computes z[0:4] = x[0:4] + y[0:4]; the second computes z[4:8] = x[4:8] + y[4:8]. Each expression describes a vectorized block operation, not necessarily a single hardware instruction. section 2.1.1
Figure 2.1 CUDA assigns one thread to each element, so two blocks of four threads (T0-T3) each compute four scalar additions individually. Triton assigns one program instance to each whole block, so program P0 computes z[0:4] = x[0:4] + y[0:4] and program P1 computes z[4:8] = x[4:8] + y[4:8] as single block-level operations.
Compare CUDA’s per-thread additions with Triton’s two tile-level program instances. section 2.1.1
Each instance uses its program ID to find its tile’s starting position, then combines that start with a vector of local indices to form global offsets. If the last tile extends beyond the input, a mask marks valid positions so tl.load and tl.store skip out-of-range accesses. The host specifies the grid; GPU hardware schedules the instances across streaming multiprocessors as resources become available. section 2.4
The abstraction does not remove all indexing or performance work. When logical coordinates differ from the physical memory layout, calculate the offsets for concrete dimensions and check that outputs are written as intended. Conditional logic can also cause warp divergence, forcing a warp to execute both branches sequentially when its threads disagree. section 3.5.4 section 2.3
