A GPU kernel is a function launched by the host CPU to run in parallel on the GPU. It reads input tensors and writes outputs without returning control to Python partway through its computation. section 1.0 section 1.3.3
A custom kernel can combine operations that eager execution would launch separately. Keeping intermediate results on-chip can reduce memory traffic and launch overhead. This is useful when profiling reveals a slow operation or a sequence of small operations that framework-level optimization does not handle well. section 1.0 section 1.1.2
In Triton, an entry point is a Python function decorated with @triton.jit. On its first call, Triton compiles it into GPU code. The host launches a grid of program instances; each instance executes the kernel for its assigned block of data. Triton manages the underlying threads and much of their coordination. section 1.1.1 section 1.3.3
Vector addition
For Z = X + Y, each program instance adds a distinct segment of the input vectors and writes the corresponding segment of Z. The host wrapper chooses a grid large enough to cover the vector. Inside the kernel, the program ID selects a block, offsets identify its elements, and a mask prevents loads and stores beyond the vector’s end. The result can be checked against PyTorch addition. section 2.2
Figure 2.3 Block-level parallel vector addition in Triton. Each Triton program (block) adds a distinct segment of input vectors X and Y, producing its corresponding slice of the output vector Z. This illustrates Triton’s block-level data parallelism model, where each program instance operates independently on disjoint data
See how program instances divide vector addition into independent segments. section 2.2
The same idea extends to matrices: a two-dimensional grid gives each program row and column coordinates for a tile. Offsets, strides, and a two-dimensional mask determine the addresses and valid elements. section 5.3.1
Performance depends on more than parallel arithmetic. Registers hold thread-private values, while shared memory lets threads within a block exchange data; global memory is larger but slower. Triton handles much of the low-level memory management, but data movement still matters. For small or heavily branching workloads, GPU launch and transfer costs may outweigh the benefit of parallel execution. section 1.4 section 1.3.2



