liveBook · Manning

Tensor Core

As explained in GPU Programming with Triton

A Tensor Core is a specialized GPU execution unit for high-throughput matrix and tensor operations.

A Tensor Core is a specialized execution unit integrated with an NVIDIA GPU’s streaming multiprocessor (SM), built for matrix multiply-accumulate operations. Unlike scalar CUDA cores, which process scalar values per active thread, a matrix instruction describes many related multiply-accumulates over a supported matrix shape. section 5.7

\[D_{i,j} = C_{i,j} + \sum_{k=0}^{K-1} A_{i,k}B_{k,j}.\]

The matrix multiply-accumulate updates each output position by accumulating products across the shared K dimension. section 5.7

A tiled Triton matrix product uses tl.dot to compute a block of outputs at once, reaching the Tensor Core execution path. This replaces a long stream of scalar multiply, add, and reduction instructions with matrix operations. section 5.7 section 5.0

Counting work in a tile

Multiplying a [64, 32] matrix by a [32, 64] matrix represents 131,072 multiply-accumulates, or 262,144 floating-point operations. section 5.7

How GPU Programming with Triton covers it

How deep it goes
Explains scalar FMA, matrix MMA shapes, arithmetic counts, and the Tensor Core path reached by tiled tl.dot.
Examples
Quantifies a [64, 32] by [32, 64] product as 131,072 multiply-accumulates and 262,144 floating-point operations.
What you can do after
Apply it via Triton's tl.dot to express a tiled matrix product for Tensor Core execution.
How it connects
Connects matrix multiplication and tl.dot to MMA, contrasting Tensor Cores with scalar CUDA cores on an SM.

Also explained in

Compare how 4 books explain Tensor Core →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.