concept equation in category deep learning

appears as: equation, equations, equation, equations
Math and Architectures of Deep Learning

This is an excerpt from Manning's book Math and Architectures of Deep Learning.

In mathematical parlance, transformation via equation 1.1, v ∈ [vmin, vmax] → vnorm ∈ [0, 1] maps the values v from the input domain [vmin, vmax] to the output values vnorm in the range [0, 1].

Figure 2.2: Image corresponding to matrix I4,9 in equation 2.2

This is a non-square 15 × 3 linear system. There are only 3 unknowns to solve for, w0,w1, b and there are 15 equations. Highly redundant, we needed only 3 equations and could have solved it via Gaussian Elimination (section 2.12. But the important thing to note is this: the equations are not fully consistent. There is no single set of values for the unknown that will satisfy all of them. In other words, the training data is not fully consistent - an almost universal occurrence in all real life machine learning systems. Consequently, we have to find a solution that is optimal (causes as little error as possible) over all the equations.