Matrix-matrix multiplication combines a matrix A with m rows and p columns and a compatible matrix B with p rows and n columns, producing C = AB with m rows and n columns. Compatibility requires the left matrix’s column count to equal the right matrix’s row count. Each element Cᵢⱼ is the dot product of row i of A and column j of B. section 2.5.3
A small product
For A = [[1, 2], [3, 4], [5, 6]] and B = [[7, 8], [9, 10]], torch.matmul(A, B) computes each output entry from one row of A and one column of B. For example, C₂₁ = 3·7 + 4·9 = 57, and the result has three rows and two columns. section 2.5.3 section 2.7.4
The operation generalizes matrix-vector multiplication: a matrix transforms a column vector, while multiplying two matrices applies the same row-column calculation across many vectors at once. In machine learning, this packs weighted sums for many data instances into compact equations and connects finite-dimensional matrix multiplication with linear transformations. section 2.10.2 section 2.5.3
Important identities and constraints include:
- Do not reverse factors casually: matrix multiplication is not commutative, so AB generally differs from BA. section 2.5.3
- Transpose reverses the product order: (AB)ᵀ = BᵀAᵀ. section 2.7.5
- View one-dimensional convolution as matrix-vector multiplication using a sparse block-diagonal weight matrix whose shifted rows encode kernel positions. section 10.1.3
Listing 2.8 Matrix-matrix multiplication
A = torch.tensor([[1, 2], [3, 4], [5, 6]]) B = torch.tensor([[7, 8], [9, 10]]) C = torch.matmul(A, B) ① ② w = torch.tensor([1, 2, 3]) x = torch.tensor([4, 5, 6]) assert torch.dot(w, x) == torch.matmul(w.T, x) ③ ① C = AB ⟹ C[i, j] is the dot product of the ith row of A and jth column of B. ② ③ The dot product can be viewed as a row matrix multiplied by a column matrix. |
The PyTorch listing makes the row-column rule executable and contrasts matrix multiplication with the vector dot product. section 2.7.4
