liveBook · Manning

Matrix multiply-accumulate

A matrix multiply-accumulate operation multiplies two matrices and adds the product to an accumulator matrix.

How 2 Manning books explain it

GPU Programming with Triton

A matrix multiply-accumulate operation multiplies two matrices and adds a third matrix to the product.

How it is used
Explains why dense matrix multiplication maps to matrix instructions rather than streams of scalar arithmetic.
How deep it goes
Defines the operation, shapes, and tiled accumulation, while leaving layout and tile requirements target-dependent.
Examples
Compares scalar FMA with a [64, 32] by [32, 64] workload and shows accumulation across K slices.
What you can do after
Apply it via Triton's tl.dot to accumulate tiled products across K slices and produce an output tile.

Read how GPU Programming with Triton explains matrix multiply-accumulate →

CUDA for LLMs

Matrix multiply-accumulate multiplies matrices and adds the resulting product to an accumulator.

How it is used
Maps matrix-product accumulation onto tensor-core hardware when computation decomposes into suitable matrix patterns.
How deep it goes
Defines D = A × B + C, explains mixed precision and warp-level WMMA tiles, and notes shape and layout constraints.
Examples
Uses D = A × B + C, scaled GEMM, and 16 × 16 × 16 WMMA tiles accumulated across K.
What you can do after
Apply CUDA WMMA to accumulate tiled matrix products with mma_sync while accounting for fragment-loading overhead and warp work.

Read how CUDA for LLMs explains matrix multiply-accumulate →

What to know first

Related concepts

Questions about matrix multiply-accumulate

What is matrix multiply-accumulate?

A matrix multiply-accumulate operation multiplies two matrices and adds the product to an accumulator matrix.

Which Manning books explain matrix multiply-accumulate?

GPU Programming with Triton and CUDA for LLMs.

What should I know before learning matrix multiply-accumulate?

Manning books that teach matrix multiply-accumulate usually introduce bfloat16, float16 and tile first.

What is matrix multiply-accumulate a kind of?

Matrix multiply-accumulate is a kind of multiply-accumulate. A multiply-accumulate operation multiplies two values and adds their product to an accumulated result.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.