liveBook · Manning

Matrix-matrix multiplication

As explained in Build a DeepSeek Model (From Scratch)

Matrix multiplication is an algebraic operation that combines rows of one matrix with columns of another to produce a matrix.

Matrix-matrix multiplication combines rows from one matrix with columns from another, producing a new matrix. In transformer attention, two large products dominate the computation: the query-key product QKᵀ and the attention-weights-times-value product. Repeating these products for every generated token creates the quadratic inference bottleneck in sequence length. section 2.3.1

The same operation also forms the computational core of neural-network linear layers and their gradients:

  • Forward propagation computes output from weights and input, written conceptually as y = W × x. section 5.6.1
  • Input and weight values can be converted to FP8 for fast multiplication, while outputs are accumulated with higher precision. section 5.6.1
  • The input gradient uses dL/dx = (dL/dz) × Wᵀ, while the weight gradient uses dL/dW = xᵀ × (dL/dz). section 5.6.1
  • Master weights remain in FP32 or BF16, and the weight update is performed in FP32 to preserve training stability. section 5.6.1
Figure 5.22 The mixed-precision framework with FP8 data format in the Linear operator. The diagram illustrates the flow of data through a linear layer, showing how inputs and weights are converted to low-precision FP8 for fast computation, while critical components like master weights and gradients are maintained in high-precision FP32 for stability.

Trace how a linear layer converts inputs and weights to FP8 for multiplication while retaining high-precision weights and gradients. section 5.6.1

How Build a DeepSeek Model (From Scratch) covers it

How deep it goes
Connects forward, backward, precision, and update paths, but stops short of a general algorithm or implementation listing.
Examples
Uses QKᵀ, attention-weights-times-value, forward propagation, Dgrad, Wgrad, and FP32 weight-update equations.
What you can do after
Apply it via mixed-precision linear layers and identify the matrix products driving attention cost.
How it connects
Connects attention, inference, linear layers, mixed precision, FP8 GEMMs, gradients, and master weights.

Also explained in

Compare how 13 books explain matrix-matrix multiplication →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build a DeepSeek Model (From Scratch). The section links point to the parts of the book it draws on.