tl.dot computes a matrix product over input blocks. For two-dimensional tiles, [M, K] × [K, N] produces [M, N]: the matching K dimensions are contracted, while M and N determine the output shape. With an accumulator, tl.dot(a, b, acc) returns a @ b + acc. section 5.6.1
In a tiled matrix-multiplication kernel, each call handles one K slice and adds its product to an FP32 accumulator. The compiler uses tl.dot to coordinate layouts, register allocation, and latency hiding; loads, pointer traversal, boundary handling, and stores remain explicit in the surrounding kernel. Compatible FP16 and BF16 tiles can expose a Tensor Core execution path, subject to backend requirements. section 5.5.5 section 5.6.1 section 5.10
Keep these implementation constraints in view:
- The input tiles must have matching K dimensions; target backends may impose further layout, dtype, or tile-size requirements. section 5.6.1
- A first tiled kernel may assume dimensions divide evenly by tile sizes. A boundary-safe version masks incomplete M and N tiles and makes invalid K positions contribute zero. section 5.5.5 section 5.10
Listing 5.8 Tiled matmul kernel, assuming dimensions are exactly divisible by the tile sizes
import triton
import triton.language as tl
@triton.jit
def tiled_matmul_kernel(
a_ptr, b_ptr, c_ptr,
M, N, K,
stride_am, stride_ak,
stride_bk, stride_bn,
stride_cm, stride_cn,
BLOCK_M: tl.constexpr, #1
BLOCK_N: tl.constexpr,
BLOCK_K: tl.constexpr,
):
pid_m = tl.program_id(axis=0) #2
pid_n = tl.program_id(axis=1)
offs_m = (
pid_m * BLOCK_M + tl.arange(0, BLOCK_M)
) #3
offs_n = pid_n * BLOCK_N + tl.arange(0, BLOCK_N)
offs_k = tl.arange(0, BLOCK_K)
a_ptrs = (
a_ptr
+ offs_m[:, None] * stride_am
+ offs_k[None, :] * stride_ak
) #4
b_ptrs = (
b_ptr
+ offs_k[:, None] * stride_bk
+ offs_n[None, :] * stride_bn
) #4
acc = tl.zeros(
(BLOCK_M, BLOCK_N), dtype=tl.float32
) #5
for k_start in range(0, K, BLOCK_K): #6
a = tl.load(a_ptrs)
b = tl.load(b_ptrs)
acc = tl.dot(a, b, acc) #7
a_ptrs += BLOCK_K * stride_ak #8
b_ptrs += BLOCK_K * stride_bk
c_ptrs = (
c_ptr
+ offs_m[:, None] * stride_cm
+ offs_n[None, :] * stride_cn
) #9
tl.store(c_ptrs, acc)
See the aligned-only Triton kernel: it assigns each program an output tile, loads matching A and B blocks, accumulates with tl.dot, and stores the result. section 5.5.5
K-loop accumulation
With K = 4096 and BLOCK_K = 128, the aligned kernel performs 32 tl.dot calls, each adding a K slice to the output-tile accumulator. section 5.5.5
