How deeply does Math and Architectures of Deep Learning cover tangent plane?

Explains geometric meaning through contours, surfaces, normals, gradients, and directional derivatives without deriving optimization algorithms.

Tangent plane

As explained in Math and Architectures of Deep Learning

A line or direction that touches a curve locally and indicates the direction along which movement leaves a level contour unchanged.

A tangent plane describes directions that remain locally on a level surface, so the function value does not change to first order. For a level contour, the corresponding tangent direction follows the contour; movement perpendicular to it gives the greatest change. The gradient is normal to the tangent surface. section 3.3.2

Use the tangent–normal relationship to interpret loss landscapes:

Moving along a tangent direction stays on the same level contour or surface. section 3.3.2
Moving along the normal, which is the gradient direction, crosses level surfaces with maximum change. section 3.3.2
In model training, these directions describe how parameter movement changes or preserves the loss. section 3.3.2

Circular level contours

For L(w0, w1) = √(w0² + w1²), a level contour is a circle. Moving around its circumference follows the tangent and leaves L unchanged; moving radially follows the normal and changes L most rapidly. section 3.3.2

How Math and Architectures of Deep Learning covers it

Cover of Math and Architectures of Deep Learning

liveBook manning.com

Read Math and Architectures of Deep Learning on liveBook Get the book at manning.com

Related concepts

Sharpe ratioequivalent to · The Sharpe ratio measures an investment’s excess expected return over the risk-free rate per unit of return volatility.
planekind of · A plane is a flat two-dimensional geometric surface extending indefinitely in two independent directions.
second derivativemeasured by · A second derivative is the derivative of a function’s first derivative with respect to the same variable, describing how its rate of change varies.
locally optimal solutioncontrasts with · A locally optimal solution is a feasible point whose objective value is no worse than those of sufficiently nearby points, though a better solution may exist elsewhere.

This explainer was generated by Manning's knowledge graph from the text of Math and Architectures of Deep Learning. The section links point to the parts of the book it draws on.