In CUDA terminology, the device is the GPU that executes dispatched work; the host is the CPU that prepares inputs, chooses computations, and launches kernels. Triton kernels run on the device, but are always launched from the host. Keeping track of where data lives matters: inputs already on the GPU can be processed there, while moving data between CPU and GPU incurs transfer costs. section 1.3.2 section 1.5
Vector addition
For vector addition, the host prepares tensors and configures a grid; each device-side program instance adds a slice of the input vectors and writes its corresponding output slice. section 2.2
Figure 2.3 Block-level parallel vector addition in Triton. Each Triton program (block) adds a distinct segment of input vectors X and Y, producing its corresponding slice of the output vector Z. This illustrates Triton’s block-level data parallelism model, where each program instance operates independently on disjoint data
See how independent Triton program instances handle distinct vector segments on the device. section 2.2
