tl.sum reduces a tile along a specified dimension inside one Triton program instance. For a one-dimensional tile, tl.sum(x, axis=0) produces a scalar. The axis refers to the local tile, not the full global tensor. The optional dtype argument selects the accumulator type, as in tl.sum(x, axis=0, dtype=tl.float32). section 4.1.2
A common pattern loads a tile with tl.load and reduces the values distributed across its lanes. For an incomplete tile, mask out-of-bounds loads and use other=0.0 so those lanes do not change the sum. section 4.1.2
Listing 4.2 Two-pass global sum using partial results
import torch
import triton
import triton.language as tl
@triton.jit
def partial_sum_kernel(x_ptr, partial_ptr, n_elements,
BLOCK_SIZE: tl.constexpr):
pid = tl.program_id(0)
offsets = pid * BLOCK_SIZE + tl.arange(0, BLOCK_SIZE)
mask = offsets < n_elements
x = tl.load(x_ptr + offsets, mask=mask, other=0.0)
part = tl.sum(x, axis=0, dtype=tl.float32) #1
tl.store(partial_ptr + pid, part) #2
@triton.jit
def final_sum_kernel(partial_ptr, out_ptr, n_partials,
BLOCK_SIZE: tl.constexpr):
offsets = tl.arange(0, BLOCK_SIZE)
mask = offsets < n_partials
x = tl.load(partial_ptr + offsets, mask=mask, other=0.0) #3
total = tl.sum(x, axis=0, dtype=tl.float32) #4
tl.store(out_ptr, total)
def sum_two_pass(x: torch.Tensor, block_size: int = 1024):
assert x.is_cuda and x.ndim == 1
n = x.numel()
grid0 = (triton.cdiv(n, block_size),)
partial = torch.empty((grid0[0],), device=x.device, dtype=torch.float32)
partial_sum_kernel[grid0](x, partial, n, BLOCK_SIZE=block_size)
assert partial.numel() <= block_size #5
out = torch.empty((1,), device=x.device, dtype=torch.float32)
final_sum_kernel[(1,)](partial, out, partial.numel(),
BLOCK_SIZE=block_size)
return out[0]
The first kernel reduces each masked tile to an FP32 partial sum; the second reduces the partials. Its single final tile must cover all partials. section 4.1.2
tl.sum cannot combine values held by different program instances. For a grid-wide sum, write partial sums to global memory and reduce them in another kernel, or atomically add each partial to a shared output. The atomic example initializes that output to zero. section 4.1.2
RMS normalization
For each row, accumulate squared values in an FP32 tile, then use tl.sum to obtain the sum of squares. Compute the inverse RMS from that scalar and apply it to the row with the learned scale. Masked loads contribute zero. section 4.1.4
Fused softmax
When one program instance can cover the whole vector, it can compute the maximum, exponentiate shifted values, use tl.sum for the denominator, and normalize without staging intermediates in global memory. A larger tile also increases on-chip resource use. section 4.2.2
