Tensor Core
A Tensor Core is a specialized NVIDIA GPU execution unit designed to accelerate high-throughput matrix and tensor arithmetic.
CUDA for LLMs
A tensor core is a specialized GPU execution unit designed to accelerate matrix operations commonly used in machine learning.
- How it is used
- Accelerates dense matrix multiplication in deep-learning workloads and provides a path beyond manually optimized CUDA-core GEMM.
- How deep it goes
- Develops WMMA and Hopper WGMMA kernels through tiling and asynchronous pipelines, while deferring Blackwell TCGen05 implementation.
- Examples
- Uses a 16 × 16 × 16 WMMA tile and progressively optimized H100 GEMM kernels to demonstrate execution and throughput.
- What you can do after
- Build tensor-core GEMM kernels with WMMA or Hopper WGMMA, and evaluate when library implementations are preferable.
Build a DeepSeek Model (From Scratch)
A Tensor Core is a GPU computational unit specialized for high-throughput matrix multiplication and accumulation, typically at reduced numerical precision.
- How it is used
- Deploys low-precision matrix multiplication in GPU-based LLM computation while addressing accumulation-precision limits.
- How deep it goes
- Explains products, accumulators, underflow, and periodic precision promotion, but stops short of implementation code.
- Examples
- Uses a dot-product formula, H800 hardware, inner dimension 4096, and figures showing burst accumulation with FP32 promotion.
- What you can do after
- Evaluate accumulation-precision trade-offs and understand why periodic promotion can protect low-precision matrix results.
Read how Build a DeepSeek Model (From Scratch) explains Tensor Core →
GPU Programming with Triton
A Tensor Core is a specialized GPU execution unit for high-throughput matrix and tensor operations.
- How it is used
- Accelerates tiled matrix multiplication by executing matrix multiply-accumulate work on specialized NVIDIA GPU hardware.
- How deep it goes
- Explains scalar FMA, matrix MMA shapes, arithmetic counts, and the Tensor Core path reached by tiled tl.dot.
- Examples
- Quantifies a [64, 32] by [32, 64] product as 131,072 multiply-accumulates and 262,144 floating-point operations.
- What you can do after
- Apply it via Triton's tl.dot to express a tiled matrix product for Tensor Core execution.
Rearchitecting LLMs
A Tensor Core is a specialized NVIDIA GPU execution unit optimized for high-throughput matrix arithmetic.
- How it is used
- Guides width-pruning choices for improving GPU compute efficiency through intermediate-dimension alignment.
- How deep it goes
- Explains tiled matrix multiplication, alignment, padding, and hardware bottlenecks.
- Examples
- Uses FP16 16×16 tiles, a misaligned size of 6554, and an expansion divisor of 64.
- What you can do after
- Configure OptiPFair width pruning with an expansion_divisor to align the retained neuron count.
What to know first
- attentionAttention is a neural-network mechanism that computes input-dependent relevance weights to combine information from representations for a context-dependent computation.
- bfloat16bfloat16 is a 16-bit floating-point format with one sign bit, eight exponent bits, and seven fraction bits, providing a wide numerical range with reduced precision.
- float16float16 is a 16-bit floating-point format, commonly IEEE 754 binary16, with less precision and range than 32-bit floating-point formats.
- matrix-matrix multiplicationMatrix multiplication is an operation on compatible matrices in which each entry of the product is the dot product of a row of the first matrix and a column of the second.
Related concepts
- gpu corecontrasts with · A CUDA core is a relatively simple arithmetic and logic execution unit in an NVIDIA GPU that processes instructions for individual threads.
- streaming multiprocessorpart of · A streaming multiprocessor is a GPU execution unit that schedules and executes groups of threads using its processing and on-chip memory resources.
- cuBLASused by · cuBLAS is NVIDIA’s GPU-accelerated library of basic linear algebra operations for CUDA applications.
- fused Flash Attention kernelused by · The fused Flash Attention kernel is an example CUDA implementation that combines attention computations in one kernel.
- matrix multiply-accumulateuses · A matrix multiply-accumulate operation multiplies two matrices and adds the product to an accumulator matrix.
Questions about Tensor Core
What is Tensor Core?
A Tensor Core is a specialized NVIDIA GPU execution unit designed to accelerate high-throughput matrix and tensor arithmetic.
Which Manning books explain Tensor Core?
CUDA for LLMs, Build a DeepSeek Model (From Scratch), GPU Programming with Triton and Rearchitecting LLMs.
What should I know before learning Tensor Core?
Manning books that teach Tensor Core usually introduce attention, bfloat16, float16 and matrix-matrix multiplication first.
What is the difference between Tensor Core and gpu core?
A Tensor Core is a specialized NVIDIA GPU execution unit designed to accelerate high-throughput matrix and tensor arithmetic. A CUDA core is a relatively simple arithmetic and logic execution unit in an NVIDIA GPU that processes instructions for individual threads.
What is Tensor Core part of?
Tensor Core is part of streaming multiprocessor. A streaming multiprocessor is a GPU execution unit that schedules and executes groups of threads using its processing and on-chip memory resources.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.



