Matrix-matrix multiplication
Matrix multiplication is an operation on compatible matrices in which each entry of the product is the dot product of a row of the first matrix and a column of the second.
CUDA for LLMs
General matrix multiplication (GEMM) computes a scaled matrix product and accumulates it into a scaled existing matrix.
- How it is used
- Uses GEMM for dense layers and batched transformer processing, contrasting it with single-token matrix-vector multiplication.
- How deep it goes
- Develops GEMM from dot-product loops through CUDA tiling, register reuse, and profiling, then distinguishes tensor-core accumulation from scaled GEMM.
- Examples
- Works through a linear layer’s forward and backward products and profiles FP16 GEMM variants on 4096 × 4096 matrices.
- What you can do after
- Builds a plain matrix product in C and CUDA and applies scaled GEMM through cuBLAS with attention to layout.
Read how CUDA for LLMs explains matrix-matrix multiplication →
Build AI Drug Discovery Pipelines
Matrix multiplication is the arithmetic operation that combines matrices to produce a matrix of weighted linear transformations.
- How it is used
- Applies matrix multiplication to tensor computations within PyTorch deep-learning workflows.
- How deep it goes
- Explains the operators, shape constraint, transpose support, and worked output, but not the underlying arithmetic algorithm.
- Examples
- Demonstrates multiplying a two-dimensional tensor by its transpose with matmul() and @.
- What you can do after
- Apply it via PyTorch using torch.matmul() or @ while checking compatible tensor dimensions.
Read how Build AI Drug Discovery Pipelines explains matrix-matrix multiplication →
Grokking GPU Programming
Matrix multiplication is the algebraic operation that combines rows of one matrix with columns of another to produce a matrix of dot products.
- How it is used
- Uses matrix multiplication to motivate GPU parallelism and reducing global-memory traffic through tiling.
- How deep it goes
- Works from dot products and dimensions through thread indexing, cooperative tiling, resource limits, and a cuBLAS call.
- Examples
- Uses a 2×3 by 3×2 product, a 1024×1024 by 1024×512 workload, and a matrix multiplication exercise.
- What you can do after
- Apply it via cuBLAS or build a CUDA kernel with one thread per output and cooperative tile loading.
Read how Grokking GPU Programming explains matrix-matrix multiplication →
Geometry for Programmers
Matrix multiplication is an operation that combines matrices by taking row-column dot products to produce a new matrix or vector.
- How it is used
- Unifies geometric transformations, point projection, and linear-equation notation as number-crunching operations.
- How deep it goes
- Defines row-column products, dimension constraints, noncommutativity, vector cases, and composable projective transformations.
- Examples
- Uses point projection, quadrilateral transformation, vector-matrix products, and matrix forms of linear systems.
- What you can do after
- Apply it via transformation routines or write direct routines when framework overhead or missing operations demand it.
Read how Geometry for Programmers explains matrix-matrix multiplication →
Math and Architectures of Deep Learning
The result of multiplying two compatible matrices, here discussed under the rule that transposition reverses the order of factors.
- How it is used
- Packages many weighted sums into compact machine-learning computations and represents linear transformations in finite dimensions.
- How deep it goes
- Defines compatibility, dimensions, row-column dot products, noncommutativity, transposition, and PyTorch execution without deriving multiplication algorithms internally.
- Examples
- Uses a three-by-two times two-by-two numeric product, PyTorch tensors, weighted-sum models, and convolution weight matrices.
- What you can do after
- Apply it via PyTorch's torch.matmul and verify products, dimensions, dot products, and transpose identities.
Read how Math and Architectures of Deep Learning explains matrix-matrix multiplication →
Statistics Every Programmer Needs
Matrix multiplication is an algebraic operation that forms each result entry from the dot product of a row of one matrix and a column of another.
- How it is used
- Forecasts future state distributions by composing transition probabilities across multiple periods.
- How deep it goes
- Defines multiplication through dot products and demonstrates iterative and direct two-period computation, without developing broader algebraic theory.
- Examples
- Uses cellular-market shares, a four-state transition matrix, NumPy dot products, and matching period-2 forecasts.
- What you can do after
- Apply it via NumPy to calculate multi-period transition matrices and forecast state distributions.
Read how Statistics Every Programmer Needs explains matrix-matrix multiplication →
GPU Programming with Triton
Matrix multiplication combines two compatible matrices by taking row-column dot products.
- How it is used
- Connects dense linear algebra to GPU workloads such as neural-network layers and attention projections.
- How deep it goes
- Derives the scalar contraction, then develops tiled tl.dot kernels with masking, data reuse, and autotuning.
- Examples
- Uses a 2×3-by-3×2 numerical product, attention projections, and accumulated weight gradients.
- What you can do after
- Build it from scratch by mapping output tiles to programs, reusing input blocks, and contracting them with tl.dot.
Read how GPU Programming with Triton explains matrix-matrix multiplication →
Build a DeepSeek Model (From Scratch)
Matrix multiplication is an algebraic operation that combines rows of one matrix with columns of another to produce a matrix.
- How it is used
- Targets the dominant matrix products in attention inference and the large multiplications inside neural-network linear layers.
- How deep it goes
- Connects forward, backward, precision, and update paths, but stops short of a general algorithm or implementation listing.
- Examples
- Uses QKᵀ, attention-weights-times-value, forward propagation, Dgrad, Wgrad, and FP32 weight-update equations.
- What you can do after
- Apply it via mixed-precision linear layers and identify the matrix products driving attention cost.
Read how Build a DeepSeek Model (From Scratch) explains matrix-matrix multiplication →
Learn Concurrent Programming with Go
Matrix multiplication is the linear-algebra operation that computes a matrix product from weighted sums of rows and columns.
- How it is used
- Uses matrix multiplication to demonstrate parallel computation and barrier synchronization in Go.
- How deep it goes
- Explains the cubic algorithm, row-column calculation, faster alternatives, and a barrier-based concurrent implementation.
- Examples
- Uses 3 × 3 matrices, random entries from −5 through 4, row workers, and printed multiplication results.
- What you can do after
- Build it from scratch using nested loops, then parallelize rows with goroutines and coordinate them with barriers.
Read how Learn Concurrent Programming with Go explains matrix-matrix multiplication →
Inside Deep Learning: Math, Algorithms, Models
The binary operation that multiplies two matrices to produce a new matrix, written with the @ operator in Python and used here for performance comparison.
- How it is used
- Uses matrix multiplication to compare CPU and GPU performance for a common neural-network operation.
- How deep it goes
- Defines output dimensions and shows a timed PyTorch operation, without deriving the multiplication algorithm.
- Examples
- Benchmarks 100 multiplications of a 2048×2048 matrix with timeit.
- What you can do after
- Evaluate CPU and GPU runtimes by timing repeated matrix multiplications and considering data-transfer overhead.
Read how Inside Deep Learning: Math, Algorithms, Models explains matrix-matrix multiplication →
Build a Robo-Advisor with Python (From Scratch)
Matrix multiplication is an algebraic operation that combines rows of one matrix with columns of another to produce a new matrix or vector.
- How it is used
- Computes portfolio expected returns, covariance matrices, and standard deviations during portfolio construction.
- How deep it goes
- Explains vector and covariance expressions with executable NumPy examples, while stopping short of broader linear-algebra theory.
- Examples
- Uses FE, WMT, and AAPL weights, asset statistics, covariance construction, and portfolio standard-deviation calculations.
- What you can do after
- Apply it via NumPy to calculate portfolio returns, covariance matrices, and risk measures without explicit loops.
Read how Build a Robo-Advisor with Python (From Scratch) explains matrix-matrix multiplication →
Deep Learning with R, Third Edition
matmul is an operation that computes matrix or batched tensor multiplication.
- How it is used
- Uses matrix products to describe geometric transforms and to perform lower-precision inference.
- How deep it goes
- Works through rotation and scaling, then details scaled int8 multiplication and output unscaling.
- Examples
- Uses 2D rotation and scaling, plus small matrices for an int8 quantization example.
- What you can do after
- Understand how scaling and unscaling let int8 matmul approximate a float32 result.
Read how Deep Learning with R, Third Edition explains matrix-matrix multiplication →
Deep Learning with Python, Third Edition
The linear-algebra operation that multiplies matrices or higher-rank tensors in a way that composes transformations.
- How it is used
- Deploys matrix multiplication as the core transformation inside dense neural-network layers and linear classifiers.
- How deep it goes
- Explains vector, matrix-vector, matrix-matrix, and higher-rank products with shapes and naive implementations, without deriving formal notation.
- Examples
- Uses NumPy vector and matrix implementations, a Dense-layer expression, and a PyTorch linear-model forward pass.
- What you can do after
- Apply it via NumPy, Keras, or PyTorch while checking operand shapes and interpreting the resulting dimensions.
Read how Deep Learning with Python, Third Edition explains matrix-matrix multiplication →
Also covered in
- Build a Large Language Model (From Scratch)
- Domain-Specific Small Language Models
- Grokking AI Algorithms, Second Edition
- Quantization and Fast InferenceIn early access (MEAP)
- Digital Twins in Action
- Type-Driven Development with Idris
- Grokking Concurrency
What to know first
- arrayAn array is an ordered collection of elements organized in one or more dimensions and accessible by position or index.
- PythonPython is a high-level, general-purpose, dynamically typed programming language with readable syntax and support for multiple programming paradigms.
- inner productAn inner product is a scalar-valued operation on two vectors that generalizes the dot product, which for real coordinate vectors is the sum of products of corresponding components.
- float16float16 is a 16-bit floating-point format, commonly IEEE 754 binary16, with less precision and range than 32-bit floating-point formats.
Related concepts
- compute-boundkind of · A computation is compute-bound when its performance is limited primarily by processing capacity rather than memory transfer, input/output, or other resources.
- cuBLASuses · cuBLAS is NVIDIA’s GPU-accelerated library of basic linear algebra operations for CUDA applications.
- attention headused by · An attention head is an independently parameterized attention computation that uses queries, keys, and values to weight and combine information across input elements.
- inner producthas part · An inner product is a scalar-valued operation on two vectors that generalizes the dot product, which for real coordinate vectors is the sum of products of corresponding components.
Questions about matrix-matrix multiplication
What is matrix-matrix multiplication?
Matrix multiplication is an operation on compatible matrices in which each entry of the product is the dot product of a row of the first matrix and a column of the second.
Which Manning books explain matrix-matrix multiplication?
CUDA for LLMs, Build AI Drug Discovery Pipelines, Grokking GPU Programming, Geometry for Programmers, Math and Architectures of Deep Learning, Statistics Every Programmer Needs, GPU Programming with Triton, Build a DeepSeek Model (From Scratch) and 5 more.
What should I know before learning matrix-matrix multiplication?
Manning books that teach matrix-matrix multiplication usually introduce array, Python, inner product and float16 first.
What is matrix-matrix multiplication a kind of?
Matrix-matrix multiplication is a kind of compute-bound. A computation is compute-bound when its performance is limited primarily by processing capacity rather than memory transfer, input/output, or other resources.
What are the parts of matrix-matrix multiplication?
Matrix-matrix multiplication includes inner product.
What is matrix-matrix multiplication also called?
Matrix-matrix multiplication is also called matmul, matrix product, GEMM or @ operator.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.












