liveBook · Manning

Block-level programming

As explained in GPU Programming with Triton

A block-level programming language lets a programmer express a GPU kernel from the perspective of one block processing one tile.

Block-level programming expresses a GPU kernel from the perspective of one program instance processing a tile of data. A launch grid creates instances for the full workload, and each instance applies the same kernel logic to its own slice. This is also called blocked or tiled SPMD (single program, multiple data). section 2.1.1 section 2.1.2

In Triton, the programmer chooses the tile size and writes vectorized loads, computation, stores, and masks for the tile. The compiler and runtime handle the distribution of work among threads within a block. By contrast, CUDA code is written from a thread’s perspective and requires explicit thread indexing and coordination. section 1.5 section 2.1.1

Vector addition in two tiles

For vectors x and y of length 8, choose a block size of 4 and launch two program instances. The first computes z[0:4] = x[0:4] + y[0:4]; the second computes z[4:8] = x[4:8] + y[4:8]. Each expression describes a vectorized block operation, not necessarily a single hardware instruction. section 2.1.1

Figure 2.1 CUDA assigns one thread to each element, so two blocks of four threads (T0-T3) each compute four scalar additions individually. Triton assigns one program instance to each whole block, so program P0 computes z[0:4] = x[0:4] + y[0:4] and program P1 computes z[4:8] = x[4:8] + y[4:8] as single block-level operations.
CH02 F01 Fartale

Compare CUDA’s per-thread additions with Triton’s two tile-level program instances. section 2.1.1

Each instance uses its program ID to find its tile’s starting position, then combines that start with a vector of local indices to form global offsets. If the last tile extends beyond the input, a mask marks valid positions so tl.load and tl.store skip out-of-range accesses. The host specifies the grid; GPU hardware schedules the instances across streaming multiprocessors as resources become available. section 2.4

The abstraction does not remove all indexing or performance work. When logical coordinates differ from the physical memory layout, calculate the offsets for concrete dimensions and check that outputs are written as intended. Conditional logic can also cause warp divergence, forcing a warp to execute both branches sequentially when its threads disagree. section 3.5.4 section 2.3

How GPU Programming with Triton covers it

How deep it goes
Works through grid launch, program IDs, offsets, masked memory operations, and scheduling; notes that block-level syntax does not eliminate divergence.
Examples
Uses vector addition over two four-element tiles and coordinate mapping with different input and auxiliary tensor strides.
What you can do after
Apply it via Triton to write and launch a masked, tiled vector-addition kernel.
How it connects
Connects tiled SPMD with vectorization, GPU scheduling, memory masking, and warp divergence.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.