liveBook · Manning

Matrix-matrix multiplication

As explained in Deep Learning with R, Third Edition

matmul is an operation that computes matrix or batched tensor multiplication.

Matrix multiplication (matmul) multiplies matrices and serves as the linear part of geometric transformations. A 2 × 2 matrix product can rotate or scale a point in 2D space. section 2.3.5

In a Dense layer, the matrix product contributes to an affine transformation; adding a vector supplies its translation. Repeated affine layers without activations still reduce to one affine transformation. section 2.3.5

Int8 quantization

To approximate a float32 matrix product with int8 arithmetic, scale each input to fit the int8 range, multiply, then unscale the output. Rounding during conversion can cause some accuracy loss. section 18.3.6

x <- op_array(rbind(c(0.1, 0.9), c(1.2, -0.8)))
kernel <- op_array(rbind(c(-0.1, -2.2), c(1.1, 0.7)))

The example provides two small matrices used as inputs to the quantized product. section 18.3.6

How Deep Learning with R, Third Edition covers it

How deep it goes
Works through rotation and scaling, then details scaled int8 multiplication and output unscaling.
Examples
Uses 2D rotation and scaling, plus small matrices for an int8 quantization example.
What you can do after
Understand how scaling and unscaling let int8 matmul approximate a float32 result.
How it connects
Connects matrix multiplication to linear transformations, affine transformations, Dense layers, and quantization.

Also explained in

Compare how 8 books explain matrix-matrix multiplication →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with R, Third Edition. The section links point to the parts of the book it draws on.