Matrix multiply-accumulate
A matrix multiply-accumulate operation multiplies two matrices and adds the product to an accumulator matrix.
GPU Programming with Triton
A matrix multiply-accumulate operation multiplies two matrices and adds a third matrix to the product.
- How it is used
- Explains why dense matrix multiplication maps to matrix instructions rather than streams of scalar arithmetic.
- How deep it goes
- Defines the operation, shapes, and tiled accumulation, while leaving layout and tile requirements target-dependent.
- Examples
- Compares scalar FMA with a [64, 32] by [32, 64] workload and shows accumulation across K slices.
- What you can do after
- Apply it via Triton's tl.dot to accumulate tiled products across K slices and produce an output tile.
Read how GPU Programming with Triton explains matrix multiply-accumulate →
CUDA for LLMs
Matrix multiply-accumulate multiplies matrices and adds the resulting product to an accumulator.
- How it is used
- Maps matrix-product accumulation onto tensor-core hardware when computation decomposes into suitable matrix patterns.
- How deep it goes
- Defines D = A × B + C, explains mixed precision and warp-level WMMA tiles, and notes shape and layout constraints.
- Examples
- Uses D = A × B + C, scaled GEMM, and 16 × 16 × 16 WMMA tiles accumulated across K.
- What you can do after
- Apply CUDA WMMA to accumulate tiled matrix products with mma_sync while accounting for fragment-loading overhead and warp work.
Read how CUDA for LLMs explains matrix multiply-accumulate →
What to know first
- bfloat16bfloat16 is a 16-bit floating-point format with one sign bit, eight exponent bits, and seven fraction bits, providing a wide numerical range with reduced precision.
- float16float16 is a 16-bit floating-point format, commonly IEEE 754 binary16, with less precision and range than 32-bit floating-point formats.
- tileA tile is a bounded block of elements from a larger array, matrix, or tensor that is treated as a unit of computation.
Related concepts
- Tensor Coreused by · A Tensor Core is a specialized NVIDIA GPU execution unit designed to accelerate high-throughput matrix and tensor arithmetic.
- tl.dotproduced by · tl.dot is a Triton operation that computes the dot product of compatible input blocks, typically as matrix multiplication.
- matrix-matrix multiplicationuses · Matrix multiplication is an operation on compatible matrices in which each entry of the product is the dot product of a row of the first matrix and a column of the second.
- mmaimplemented by · MMA is a CUDA tensor-core matrix-multiply-and-accumulate programming interface or instruction family.
Questions about matrix multiply-accumulate
What is matrix multiply-accumulate?
A matrix multiply-accumulate operation multiplies two matrices and adds the product to an accumulator matrix.
Which Manning books explain matrix multiply-accumulate?
GPU Programming with Triton and CUDA for LLMs.
What should I know before learning matrix multiply-accumulate?
Manning books that teach matrix multiply-accumulate usually introduce bfloat16, float16 and tile first.
What is matrix multiply-accumulate a kind of?
Matrix multiply-accumulate is a kind of multiply-accumulate. A multiply-accumulate operation multiplies two values and adds their product to an accumulated result.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.

