Matrix multiplication is a common GPU workload, from neural-network training to 3D rendering. Each result is built by pairing a row from the first matrix with a column from the second, multiplying corresponding values, and adding the products. section 12.0 section 12.1
If A has M rows and K columns, B must have K rows and N columns. The result C has M rows and N columns. Each C[i][j] is a dot product of K pairs; counting each multiplication and addition separately gives 2 × M × N × K floating-point operations. section 12.1
One result at a time
Multiply a 2×3 matrix A by a 3×2 matrix B to get a 2×2 matrix C. For C[0][0], pair row 0 of A with column 0 of B: 1×7 + 2×9 + 3×11 = 58. Repeat with the other row–column pairs to fill C. section 12.1
Figure 12.1 Multiplying a 2×3 matrix A by a 3×2 matrix B gives a 2×2 result C. Each element of C is the dot product of one row of A and one column of B: row 0 of A and column 0 of B (shaded) produce C[0][0] = 1×7 + 2×9 + 3×11 = 58. The remaining three elements of C come from the other row–column pairs.

Follow the highlighted row and column to see how their dot product becomes one result element. section 12.1
Each output element can be computed independently, so a simple GPU kernel assigns one thread to each element. The thread walks through the K pairs, accumulates their products, and writes its result. A two-dimensional grid maps the x direction to columns and y to rows; bounds checks protect the edges when dimensions do not fill whole thread blocks. section 12.1
__global__ void matmulNaive(float* A, float* B, float* C,
int M, int N, int K) {
int row = blockIdx.y * blockDim.y + threadIdx.y;
int col = blockIdx.x * blockDim.x + threadIdx.x;
if (row < M && col < N) {
float sum = 0.0f;
for (int k = 0; k < K; k++) {
sum += A[row * K + k] * B[k * N + col];
}
C[row * N + col] = sum;
}
}Inspect the row and column indexing, dot-product loop, and bounds check in the naive CUDA kernel. section 12.1
The simple kernel repeatedly fetches values that other threads also need. Tiling reduces that traffic: threads in a block cooperatively load pieces of A and B into shared memory, then reuse those values while computing their own outputs. Shared memory acts like a nearby whiteboard, avoiding repeated trips to global memory. section 12.0 section 12.13
A larger tile is not automatically better: it must fit both the thread-block limit and the shared-memory budget. In a one-element-per-thread tiled kernel, a 64×64 tile needs 4,096 threads and exceeds the stated 1,024-thread block cap; a 32×32 tile meets that limit. section 12.13
For production CUDA code, cuBLAS provides optimized matrix multiplication. Its single-precision cublasSgemm routine computes C = αAB + βC; use α = 1 and β = 0 for a simple product. cuBLAS assumes column-major storage, so tightly packed row-major matrices require swapping the input order and dimensions using the transpose identity. Performance depends on the GPU, matrix sizes, and precision. section 12.9
#include <cublas_v2.h>
cublasHandle_t handle;
cublasCreate(&handle);
float alpha = 1.0f, beta = 0.0f;
cublasSgemm(handle,
CUBLAS_OP_N, CUBLAS_OP_N, #A
N, M, K, #B
&alpha,
d_B, N, #C
d_A, K, #D
&beta,
d_C, N); #E
cublasDestroy(handle);See the cuBLAS call for tightly packed row-major matrices, including the swapped inputs and dimensions. section 12.9












