liveBook · Manning

Host

As explained in GPU Programming with Triton

In GPU computing, the host is the CPU-side execution environment that launches and orchestrates device work.

In GPU computing, the host is the CPU that runs the main application and coordinates work; the device is the GPU that executes it. A Triton kernel runs on the device but is launched from the host. section 1.3.2 section 1.5

For a kernel launch, host-side code prepares inputs, chooses the work to perform, computes the launch grid, and passes parameters to Triton. The GPU then runs a program instance for each grid position; when those programs finish, the output is ready. section 2.2 section 2.2.9

Launching vector addition

A host-side wrapper prepares the input tensors, computes a grid for the vector length and block size, and invokes a Triton kernel. Each device program adds its slice of the input vectors and writes the corresponding output. section 2.2

Listing 2.1 Vector addition kernel and host wrapper in Triton
import torch
import triton
import triton.language as tl


@triton.jit                               #1
def add_kernel(x_ptr, y_ptr, z_ptr,
               n_elements,
               BLOCK_SIZE: tl.constexpr): #2
    pid = tl.program_id(0)                #3
    block_start = pid * BLOCK_SIZE
    offsets = block_start + tl.arange(0,
                                     BLOCK_SIZE)   #4
    mask = offsets < n_elements

    x = tl.load(x_ptr + offsets,
                mask=mask)                #5
    y = tl.load(y_ptr + offsets,
                mask=mask)                #5
    tl.store(z_ptr + offsets,
             x + y,
             mask=mask)                   #6


def add(x: torch.Tensor,
        y: torch.Tensor,
        BLOCK_SIZE: int = 1024):          #7
    assert x.is_cuda and y.is_cuda, \
        "Inputs must be on CUDA device"
    assert x.shape == y.shape, \
        "Input shapes must match"

    z = torch.empty_like(x)
    n_elements = x.numel()

    grid = lambda meta: (                 #8
        triton.cdiv(n_elements,
                    meta['BLOCK_SIZE']),
    )

    add_kernel[grid](x, y, z,
                     n_elements,
                     BLOCK_SIZE=BLOCK_SIZE) #9
    return z


if __name__ == "__main__":                #10
    torch.manual_seed(0)
    size = 98432
    x = torch.rand(size,
                   device="cuda",
                   dtype=torch.float32)
    y = torch.rand(size,
                   device="cuda",
                   dtype=torch.float32)

    output_torch = x + y
    output_triton = add(x, y)

    max_diff = torch.max(
        torch.abs(output_torch
                  - output_triton)
    ).item()

    print(f"Maximum absolute "
          f"difference: {max_diff:.3e}")
    assert torch.allclose(output_triton,
                          output_torch)
    print("OK: Triton result "
          "matches PyTorch.")

Inspect the complete vector-addition listing to see the device kernel alongside its host-side wrapper. section 2.2

How GPU Programming with Triton covers it

How deep it goes
Traces launch parameters through grid execution and output completion, including program IDs, offsets, masks, loads, and stores.
Examples
Uses PyTorch matrix multiplication and a Triton vector-addition kernel with a host-side launch wrapper.
What you can do after
Operate a Triton launch by preparing inputs, computing a grid, and invoking a device kernel from host-side code.
How it connects
Connects CPU, GPU, CUDA, Triton kernels, PyTorch, and host-to-device data movement.

Also explained in

Compare how 2 books explain host →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.