A GPU runtime is the backend software layer that lets Triton kernels reach a physical GPU. Triton keeps the kernel language and compiler inside Python, but execution still depends on a matching runtime: CUDA 11.8 or later on NVIDIA hardware, or ROCm on AMD hardware. section 1.1.1
The runtime supports the transition from a launched kernel to completed GPU work. Triton supplies program IDs, block sizes, and offsets; the launch grid and runtime schedule program instances across available streaming multiprocessors. section 2.2.9
A runtime mismatch or missing backend prevents execution even when the Python package is installed:
- NVIDIA kernels compile to PTX through CUDA. section 1.1.1
- AMD kernels compile to AMDGPU IR through ROCm. section 1.1.1
- The runtime schedules the resulting program instances without requiring the kernel to know the input length or GPU-wide block placement. section 2.2.9
Vector kernel scheduling
For a vector containing 98,432 elements and a block size of 1,024, the launch grid creates 97 blocks. The runtime assigns these programs to streaming multiprocessors and continues dispatching work until the output vector is complete. section 2.2.9
