liveBook · Manning

Tensor

As explained in GPU Programming with Triton

A tensor is a multidimensional array used to represent and manipulate structured data.

A tensor is a multidimensional array of numbers. In Triton, kernels operate on tensor tiles—chunks processed by individual program instances—rather than expressing work one scalar thread at a time. The compiler maps tile operations onto GPU hardware. section 1.1.1

A matrix is two-dimensional in the computation but linear in memory. Its shape, strides, tile dimensions, and program ordering determine which values are loaded together and reused on-chip. section 5.0

A Triton reduction operates on the tl.tensor held by one program instance: its axis reduces a tile dimension, not the full tensor. A grid-wide reduction needs additional orchestration, such as a second kernel or atomics. section 4.3

Tensor operations appear in GPU workloads such as:

  • Matrix multiplication, where tiled data feeds a tl.dot kernel. section 5.0
  • Reductions, including sums, maxima, and custom combinations over a tile. section 4.3

How GPU Programming with Triton covers it

How deep it goes
Explains tile execution, matrix memory layout, and local reductions, while noting that grid-wide reductions require extra orchestration.
Examples
Uses matrix multiplication and tile-local sum, maximum, and custom reductions as examples.
What you can do after
Understand how Triton maps tensor tiles to GPU work and why matrix layout and reduction scope matter.
How it connects
Connects tensors to Triton, GPU programming, matrix multiplication, and reductions.

Also explained in

Compare how 10 books explain tensor →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of GPU Programming with Triton. The section links point to the parts of the book it draws on.