liveBook · Manning

Matrix multiplication kernel

As explained in GPU Programming with Triton

A matrix multiplication kernel is a GPU kernel that computes a matrix product.

A matrix multiplication kernel computes C = AB on a GPU. Matrix products underpin linear layers and attention projections, and can also express work in some convolution implementations. Performance depends not just on correct indexing, but on how data is accessed, reused, and mapped to GPU hardware. section 5.0 section 5.10

A practical implementation path moves from correctness to performance:

  • Start with one program per output element. Accumulate its dot product over K in chunks, using explicit strides so inputs with different memory layouts can be addressed. section 5.5.2
  • Move to tiled computation: each program computes an output tile and reuses loaded input values. Use tl.dot to contract K tiles into an FP32 accumulator. section 5.10
  • Handle incomplete tiles with predicates; invalid K positions must contribute zero. Grouped program ordering and autotuning can help optimize cache reuse and tile configuration. section 5.10

Scalar baseline

For each output C[i, j], the naive kernel loads strips from row i of A and column j of B, multiplies them elementwise, reduces the products, and accumulates across K. It promotes FP16 or BF16 operands before multiplication so the products are accumulated in FP32. section 5.5.2

Listing 5.6 Naive matrix multiplication kernel, one output element per program
import torch
import triton
import triton.language as tl

@triton.jit
def naive_matmul_kernel(
    a_ptr, b_ptr, c_ptr,                        #1
    M, N, K,                                    #2
    stride_am, stride_ak,                       #3
    stride_bk, stride_bn,
    stride_cm, stride_cn,
    BLOCK_K: tl.constexpr,                      #4
):
    pid_m = tl.program_id(axis=0)               #5
    pid_n = tl.program_id(axis=1)
    acc = 0.0                                   #6
    for k_start in range(0, K, BLOCK_K):        #7
        offs_k = k_start + tl.arange(0, BLOCK_K) #8
        a_ptrs = (
            a_ptr + pid_m * stride_am
            + offs_k * stride_ak
        ) #9
        b_ptrs = b_ptr + offs_k * stride_bk + pid_n * stride_bn
        mask_k = offs_k < K                      #10
        a = tl.load(a_ptrs, mask=mask_k, other=0.0)
        b = tl.load(b_ptrs, mask=mask_k, other=0.0)
        a_fp32 = a.to(tl.float32)
        b_fp32 = b.to(tl.float32)
        acc += tl.sum(a_fp32 * b_fp32, axis=0)   #11
    c_ptr_ij = c_ptr + pid_m * stride_cm + pid_n * stride_cn
    tl.store(c_ptr_ij, acc)                      #12

Inspect the Triton baseline that assigns one program to each output element and advances through K in blocks. section 5.5.2

How GPU Programming with Triton covers it

How deep it goes
Builds from scalar accumulation to tiled tl.dot, Tensor Core execution, boundary handling, and autotuning.
Examples
Uses a one-output-per-program Triton baseline and compares an optimized kernel with torch.matmul.
What you can do after
Build a Triton matrix-product kernel with explicit strides, K accumulation, and boundary-safe tiled computation.
How it connects
Connects matrix multiplication to memory coalescing, shared-memory tiling, Tensor Cores, and roofline analysis.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.