liveBook · Manning

Tl.dot

As explained in GPU Programming with Triton

tl.dot is a Triton operation that computes the dot product of compatible input blocks, typically as matrix multiplication.

tl.dot computes a matrix product over input blocks. For two-dimensional tiles, [M, K] × [K, N] produces [M, N]: the matching K dimensions are contracted, while M and N determine the output shape. With an accumulator, tl.dot(a, b, acc) returns a @ b + acc. section 5.6.1

In a tiled matrix-multiplication kernel, each call handles one K slice and adds its product to an FP32 accumulator. The compiler uses tl.dot to coordinate layouts, register allocation, and latency hiding; loads, pointer traversal, boundary handling, and stores remain explicit in the surrounding kernel. Compatible FP16 and BF16 tiles can expose a Tensor Core execution path, subject to backend requirements. section 5.5.5 section 5.6.1 section 5.10

Keep these implementation constraints in view:

  • The input tiles must have matching K dimensions; target backends may impose further layout, dtype, or tile-size requirements. section 5.6.1
  • A first tiled kernel may assume dimensions divide evenly by tile sizes. A boundary-safe version masks incomplete M and N tiles and makes invalid K positions contribute zero. section 5.5.5 section 5.10
Listing 5.8 Tiled matmul kernel, assuming dimensions are exactly divisible by the tile sizes
import triton
import triton.language as tl

@triton.jit
def tiled_matmul_kernel(
    a_ptr, b_ptr, c_ptr,
    M, N, K,
    stride_am, stride_ak,
    stride_bk, stride_bn,
    stride_cm, stride_cn,
    BLOCK_M: tl.constexpr, #1
    BLOCK_N: tl.constexpr,
    BLOCK_K: tl.constexpr,
):
    pid_m = tl.program_id(axis=0) #2
    pid_n = tl.program_id(axis=1)

    offs_m = (
        pid_m * BLOCK_M + tl.arange(0, BLOCK_M)
    ) #3
    offs_n = pid_n * BLOCK_N + tl.arange(0, BLOCK_N)
    offs_k = tl.arange(0, BLOCK_K)

    a_ptrs = (
        a_ptr
        + offs_m[:, None] * stride_am
        + offs_k[None, :] * stride_ak
    ) #4
    b_ptrs = (
        b_ptr
        + offs_k[:, None] * stride_bk
        + offs_n[None, :] * stride_bn
    ) #4

    acc = tl.zeros(
        (BLOCK_M, BLOCK_N), dtype=tl.float32
    ) #5

    for k_start in range(0, K, BLOCK_K): #6
        a = tl.load(a_ptrs)
        b = tl.load(b_ptrs)
        acc = tl.dot(a, b, acc) #7
        a_ptrs += BLOCK_K * stride_ak #8
        b_ptrs += BLOCK_K * stride_bk

    c_ptrs = (
        c_ptr
        + offs_m[:, None] * stride_cm
        + offs_n[None, :] * stride_cn
    ) #9
    tl.store(c_ptrs, acc)

See the aligned-only Triton kernel: it assigns each program an output tile, loads matching A and B blocks, accumulates with tl.dot, and stores the result. section 5.5.5

K-loop accumulation

With K = 4096 and BLOCK_K = 128, the aligned kernel performs 32 tl.dot calls, each adding a K slice to the output-tile accumulator. section 5.5.5

How GPU Programming with Triton covers it

How deep it goes
Defines block shapes and accumulation, while leaving memory traversal and boundary handling to the surrounding kernel.
Examples
Uses tiled matmul, including 32 K-slice updates for K=4096 and BLOCK_K=128.
What it assumes
Assumes compatible matrix-block shapes and matching K dimensions; demonstrates the move from scalar reductions to tiled products.
What you can do after
Build tiled matrix-multiplication kernels that accumulate output blocks across K slices.
How it connects
Connects matrix multiplication and tiling to Tensor Cores, FP32 accumulators, and GPU compiler layout choices.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.