liveBook · Manning

Tl.max

As explained in GPU Programming with Triton

tl.max is a Triton operation that selects the maximum value along one or more tensor dimensions, optionally returning its index.

tl.max finds the maximum along a dimension of a Triton tensor tile. Its axis applies to that tile, which belongs to one program instance; it does not combine values across program instances. A grid-wide maximum requires coordinating partial results, such as with another kernel or atomic operations. section 4.1.2

In vector softmax, a program instance can load the full vector and use tl.max to find the value subtracted before exponentiation. Keeping the reduction local lets the pipeline stay in one kernel, at the cost of using more on-chip resources for larger tiles. section 4.2.2

Listing 4.1 Row-wise sum and max using block accumulation
import torch
import triton
import triton.language as tl

@triton.jit
def row_sum_max_kernel(x_ptr, out_sum_ptr, out_max_ptr,
                       stride_x,
                       N_COLS: tl.constexpr,
                       BLOCK_SIZE: tl.constexpr):
    row = tl.program_id(0)
    x_row_ptr = x_ptr + row * stride_x

    acc_sum = tl.zeros([BLOCK_SIZE], dtype=tl.float32) #1
    acc_max = tl.full([BLOCK_SIZE], float("-inf"), dtype=tl.float32) #2
    neg_inf = tl.full([BLOCK_SIZE], float("-inf"), dtype=tl.float32)

    for off in tl.static_range(0, N_COLS, BLOCK_SIZE): #3
        cols = off + tl.arange(0, BLOCK_SIZE)
        mask = cols < N_COLS

        x = tl.load(x_row_ptr + cols, mask=mask, other=0.0).to(tl.float32)

        acc_sum += x
        acc_max = tl.maximum(acc_max, tl.where(mask, x, neg_inf)) #4

    row_sum = tl.sum(acc_sum, axis=0) #5
    row_max = tl.max(acc_max, axis=0)

    tl.store(out_sum_ptr + row, row_sum)
    tl.store(out_max_ptr + row, row_max)


def row_sum_max(x: torch.Tensor, block_size: int = 1024):
    assert x.is_cuda and x.ndim == 2
    n_rows, n_cols = x.shape

    out_sum = torch.empty((n_rows,), device=x.device, dtype=torch.float32)
    out_max = torch.empty((n_rows,), device=x.device, dtype=torch.float32)

    grid = (n_rows,)
    row_sum_max_kernel[grid](
        x, out_sum, out_max,
        x.stride(0),
        N_COLS=n_cols,
        BLOCK_SIZE=block_size,
        num_warps=4,
    )
    return out_sum, out_max

See a row-wise kernel that accumulates a maximum across blocks of columns and handles out-of-range lanes with a mask. section 4.1.2

How GPU Programming with Triton covers it

How deep it goes
Explains axis and program-instance scope, then applies the maximum in a fused softmax.
Examples
Uses masked row-wise maximum accumulation and a vector softmax maximum.
What you can do after
Apply tl.max to reduce a tile dimension or find the stabilizing maximum in a Triton softmax kernel.
How it connects
Connects reductions to Triton program instances, global-memory staging, and softmax.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.