liveBook · Manning

Matrix-vector multiplication

Matrix-vector multiplication is the linear algebra operation of multiplying a matrix by a compatible vector to produce a vector of row-wise dot products.

How 2 Manning books explain it

CUDA for LLMs

GEMV is a general matrix-vector multiplication operation with optional scaling of the product and output vector.

How it is used
Uses GEMV for single-token transformer decoding, including attention against cached keys and values.
How deep it goes
Traces a row-wise CUDA kernel and outlines coalesced access, reductions, and vectorized loads, alongside memory-bandwidth considerations.
Examples
Compares single-query and 64-query attention scores and walks through a batched thread-per-row CUDA kernel.
What you can do after
Build a basic CUDA GEMV kernel and evaluate optimization against cuBLAS using correctness checks and profiling.

Read how CUDA for LLMs explains matrix-vector multiplication →

Math and Architectures of Deep Learning

The multiplication of a matrix by a compatible vector to produce a vector; here it is the algebraic form used to express convolution layers.

How it is used
Packages many weighted sums for training datasets and neural-network linear layers.
How deep it goes
Defines row-wise dot products, dimensions, bias addition, and layer notation, then demonstrates PyTorch without detailing multiplication internals.
Examples
Uses a biased weighted-sum model, a 15-by-2 PyTorch tensor, and a neural-network layer diagram.
What you can do after
Apply it via PyTorch to compute weighted model outputs and neural-network layer values.

Read how Math and Architectures of Deep Learning explains matrix-vector multiplication →

Also covered in

Related concepts

Questions about matrix-vector multiplication

What is matrix-vector multiplication?

Matrix-vector multiplication is the linear algebra operation of multiplying a matrix by a compatible vector to produce a vector of row-wise dot products.

Which Manning books explain matrix-vector multiplication?

CUDA for LLMs and Math and Architectures of Deep Learning.

What is the difference between matrix-vector multiplication and matrix-matrix multiplication?

Matrix-vector multiplication is the linear algebra operation of multiplying a matrix by a compatible vector to produce a vector of row-wise dot products. Matrix multiplication is an operation on compatible matrices in which each entry of the product is the dot product of a row of the first matrix and a column of the second.

What is matrix-vector multiplication a kind of?

Matrix-vector multiplication is a kind of memory bound. A workload is memory-bound when its performance is limited primarily by memory access or data-transfer capacity rather than computational throughput.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.