How deeply does Math and Architectures of Deep Learning cover displacement vector?

Derives one-dimensional displacement mathematically and extends the geometry to a two-parameter surface, without implementation details.

Displacement vector

As explained in Math and Architectures of Deep Learning

A vector describing a change in position or parameters, such as the step taken in parameter space.

A displacement vector describes a change in parameter values. For a one-dimensional parameter w, δw produces a loss change δL; for sufficiently small changes, δL ≈ (dL/dw)δw. Reducing loss therefore requires moving opposite the derivative. section 3.3.1

A two-parameter step

For L(w₀,w₁)=2w₀²+3w₁² at P=(3,4), the loss is 66. The displacement vector [-12,-24]ᵀ points along the negative gradient and gives the greatest local reduction.

How Math and Architectures of Deep Learning covers it

Cover of Math and Architectures of Deep Learning

liveBook manning.com

Read Math and Architectures of Deep Learning on liveBook Get the book at manning.com

Related concepts

vectorkind of · A vector is an ordered, indexed sequence of elements, often numerical, represented as a mathematical quantity or a computational collection.
Taylor seriesused by · A Taylor series is a power series whose coefficients are determined by a function’s derivatives evaluated at a specified expansion point.
low-rank adaptationused by · Low-rank adaptation is a parameter-efficient fine-tuning method that adapts a model by learning low-rank updates to its weights, typically while keeping most original weights fixed.

This explainer was generated by Manning's knowledge graph from the text of Math and Architectures of Deep Learning. The section links point to the parts of the book it draws on.