liveBook · Manning

Device

As explained in GPU Programming with Triton

In CUDA terminology, the device is the GPU-side processor and execution environment that executes dispatched parallel work.

In CUDA terminology, the device is the GPU that executes dispatched work; the host is the CPU that prepares inputs, chooses computations, and launches kernels. Triton kernels run on the device, but are always launched from the host. Keeping track of where data lives matters: inputs already on the GPU can be processed there, while moving data between CPU and GPU incurs transfer costs. section 1.3.2 section 1.5

Vector addition

For vector addition, the host prepares tensors and configures a grid; each device-side program instance adds a slice of the input vectors and writes its corresponding output slice. section 2.2

Figure 2.3 Block-level parallel vector addition in Triton. Each Triton program (block) adds a distinct segment of input vectors X and Y, producing its corresponding slice of the output vector Z. This illustrates Triton’s block-level data parallelism model, where each program instance operates independently on disjoint data
CH02 F03 Fartale

See how independent Triton program instances handle distinct vector segments on the device. section 2.2

How GPU Programming with Triton covers it

How deep it goes
Explains host-device roles, dispatch, data residence, and block-level execution through vector addition.
Examples
Uses PyTorch matrix multiplication and Triton vector addition to illustrate device execution.
What it assumes
Introduces the CPU-GPU division before describing Triton execution, without requiring prior GPU programming knowledge.
What you can do after
Understand which processor orchestrates work, where a Triton kernel runs, and when data moves between CPU and GPU.
How it connects
Connects the device to the host, CUDA, Triton kernels, data parallelism, and GPU memory transfers.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.