liveBook · Manning

Tensor Core

A Tensor Core is a specialized NVIDIA GPU execution unit designed to accelerate high-throughput matrix and tensor arithmetic.

How 4 Manning books explain it

CUDA for LLMs

A tensor core is a specialized GPU execution unit designed to accelerate matrix operations commonly used in machine learning.

How it is used
Accelerates dense matrix multiplication in deep-learning workloads and provides a path beyond manually optimized CUDA-core GEMM.
How deep it goes
Develops WMMA and Hopper WGMMA kernels through tiling and asynchronous pipelines, while deferring Blackwell TCGen05 implementation.
Examples
Uses a 16 × 16 × 16 WMMA tile and progressively optimized H100 GEMM kernels to demonstrate execution and throughput.
What you can do after
Build tensor-core GEMM kernels with WMMA or Hopper WGMMA, and evaluate when library implementations are preferable.

Read how CUDA for LLMs explains Tensor Core →

Build a DeepSeek Model (From Scratch)

A Tensor Core is a GPU computational unit specialized for high-throughput matrix multiplication and accumulation, typically at reduced numerical precision.

How it is used
Deploys low-precision matrix multiplication in GPU-based LLM computation while addressing accumulation-precision limits.
How deep it goes
Explains products, accumulators, underflow, and periodic precision promotion, but stops short of implementation code.
Examples
Uses a dot-product formula, H800 hardware, inner dimension 4096, and figures showing burst accumulation with FP32 promotion.
What you can do after
Evaluate accumulation-precision trade-offs and understand why periodic promotion can protect low-precision matrix results.

Read how Build a DeepSeek Model (From Scratch) explains Tensor Core →

GPU Programming with Triton

A Tensor Core is a specialized GPU execution unit for high-throughput matrix and tensor operations.

How it is used
Accelerates tiled matrix multiplication by executing matrix multiply-accumulate work on specialized NVIDIA GPU hardware.
How deep it goes
Explains scalar FMA, matrix MMA shapes, arithmetic counts, and the Tensor Core path reached by tiled tl.dot.
Examples
Quantifies a [64, 32] by [32, 64] product as 131,072 multiply-accumulates and 262,144 floating-point operations.
What you can do after
Apply it via Triton's tl.dot to express a tiled matrix product for Tensor Core execution.

Read how GPU Programming with Triton explains Tensor Core →

Rearchitecting LLMs

A Tensor Core is a specialized NVIDIA GPU execution unit optimized for high-throughput matrix arithmetic.

How it is used
Guides width-pruning choices for improving GPU compute efficiency through intermediate-dimension alignment.
How deep it goes
Explains tiled matrix multiplication, alignment, padding, and hardware bottlenecks.
Examples
Uses FP16 16×16 tiles, a misaligned size of 6554, and an expansion divisor of 64.
What you can do after
Configure OptiPFair width pruning with an expansion_divisor to align the retained neuron count.

Read how Rearchitecting LLMs explains Tensor Core →

What to know first

Related concepts

Questions about Tensor Core

What is Tensor Core?

A Tensor Core is a specialized NVIDIA GPU execution unit designed to accelerate high-throughput matrix and tensor arithmetic.

Which Manning books explain Tensor Core?

CUDA for LLMs, Build a DeepSeek Model (From Scratch), GPU Programming with Triton and Rearchitecting LLMs.

What should I know before learning Tensor Core?

Manning books that teach Tensor Core usually introduce attention, bfloat16, float16 and matrix-matrix multiplication first.

What is the difference between Tensor Core and gpu core?

A Tensor Core is a specialized NVIDIA GPU execution unit designed to accelerate high-throughput matrix and tensor arithmetic. A CUDA core is a relatively simple arithmetic and logic execution unit in an NVIDIA GPU that processes instructions for individual threads.

What is Tensor Core part of?

Tensor Core is part of streaming multiprocessor. A streaming multiprocessor is a GPU execution unit that schedules and executes groups of threads using its processing and on-chip memory resources.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.