How deeply does Math and Architectures of Deep Learning cover linear system?

Defines matrix form, exact and approximate solving, singularity, and SVD-based alternatives.

Linear system

As explained in Math and Architectures of Deep Learning

A set of linear equations considered together, typically represented in matrix form and solved for unknown variables.

A linear system is a set of simultaneous linear equations treated together and often written in matrix form. The compact matrix form scales better than listing equations one by one, especially when many unknowns are involved. section 2.12

The basic goal is to find unknown values that make the matrix equation true. In the simple square, invertible case, the solution parallels the one-variable rule x = a−1b. section 2.12

Key behaviors of linear systems in this treatment:

Square, invertible systems have an exact solution through matrix inverse. section 2.12
Zero determinants make the system singular and unsolvable exactly. section 2.12.1
Near-zero determinants make the system numerically unstable. section 2.12.1
Non-square systems are handled with the Moore-Penrose pseudo-inverse or SVD-based methods. section 2.12.4 section 4.5.4

Toy training equations

A small machine learning example turns training data into equations such as 0.11w0 + 0.09w1 + b = −0.8 and similar rows for other samples. The same setup is also shown as a larger overdetermined system with many training examples. section 2.12 section 2.12.4

a₁₁x₁ + a₁₂x₂ + a₁₃x₃ + … + a_1nx_n = b₁
a₂₁x₁ + a₂₂x₂ + a₂₃x₃ + … + a_2nx_n = b₂
⁞
a_n1x₁ + a_n2x₂ + a_n3x₃ + … + a_nnx_n = b_n

Shows the general n-equation, n-unknowns layout in explicit row form. section 2.12

How Math and Architectures of Deep Learning covers it

Cover of Math and Architectures of Deep Learning

liveBook manning.com

Read Math and Architectures of Deep Learning on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Geometry for ProgrammersA linear system is a collection of linear equations whose variables must satisfy all equations simultaneously.

Compare how 2 books explain linear system →

Related concepts

singular value decompositionused by · Singular value decomposition is a matrix factorization that expresses a matrix as the product of orthogonal or unitary matrices and a diagonal matrix of nonnegative singular values.
pseudo-inverseaddressed by · A pseudoinverse is a generalized matrix inverse that provides least-squares solutions when an ordinary inverse does not exist.
Gaussian eliminationaddressed by · Gaussian elimination is a procedure that applies elementary row operations to transform a linear system or matrix into an equivalent triangular or row-reduced form.
SymPyaddressed by · SymPy is an open-source Python library for symbolic mathematics and computer algebra.

This explainer was generated by Manning's knowledge graph from the text of Math and Architectures of Deep Learning. The section links point to the parts of the book it draws on.