liveBook · Manning

GPU runtime

As explained in GPU Programming with Triton

A GPU runtime is the software layer that provides device memory management, kernel dispatch, and execution support for a GPU backend.

A GPU runtime is the backend software layer that lets Triton kernels reach a physical GPU. Triton keeps the kernel language and compiler inside Python, but execution still depends on a matching runtime: CUDA 11.8 or later on NVIDIA hardware, or ROCm on AMD hardware. section 1.1.1

The runtime supports the transition from a launched kernel to completed GPU work. Triton supplies program IDs, block sizes, and offsets; the launch grid and runtime schedule program instances across available streaming multiprocessors. section 2.2.9

A runtime mismatch or missing backend prevents execution even when the Python package is installed:

  • NVIDIA kernels compile to PTX through CUDA. section 1.1.1
  • AMD kernels compile to AMDGPU IR through ROCm. section 1.1.1
  • The runtime schedules the resulting program instances without requiring the kernel to know the input length or GPU-wide block placement. section 2.2.9

Vector kernel scheduling

For a vector containing 98,432 elements and a block size of 1,024, the launch grid creates 97 blocks. The runtime assigns these programs to streaming multiprocessors and continues dispatching work until the output vector is complete. section 2.2.9

How GPU Programming with Triton covers it

How deep it goes
Explains backend selection, low-level compilation targets, and grid scheduling without detailing runtime internals.
Examples
Uses a 98,432-element vector, 1,024-element blocks, 97 blocks, and concurrent execution across an NVIDIA H100.
What it assumes
Assumes Python and Triton familiarity while identifying the required hardware-specific runtime from zero.
What you can do after
Operate/configure Triton execution by installing the hardware-matched CUDA or ROCm runtime.
How it connects
Connects Triton, CUDA, ROCm, PTX, AMDGPU IR, GPU kernels, launch grids, program instances, and streaming multiprocessors.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.