How deeply does Deep Learning with R, Third Edition cover derivative?

Defines local slope, extends derivatives to tensor-valued functions as gradients, and applies them procedurally in mini-batch optimization.

Derivative

As explained in Deep Learning with R, Third Edition

A derivative is a measure of how a function's output changes in response to an infinitesimal change in an input.

A derivative measures how a function’s output changes when its input changes infinitesimally. Near a point, it approximates a smooth function with a line whose slope gives the derivative: positive values indicate increase, negative values indicate decrease, and magnitude indicates rate. section 2.4.1

From derivatives to model updates

For a scalar function, the derivative points toward how the output changes. In a neural network, model weights W produce predictions and a loss value. The gradient of that loss with respect to W shows how each weight affects the loss, so updating W in the opposite direction reduces the loss. section 2.4.2 section 2.4.3

Derivatives support optimization through this progression:

Treat the loss as a function of the model’s parameters. section 2.4.2
Compute its gradient during the backward pass. section 2.4.3
Move parameters opposite the gradient, scaled by the learning rate. section 2.4.3

f(x + epsilon_x) = y + a * epsilon_x

The local approximation f(x + epsilon_x) = y + a * epsilon_x makes the derivative’s slope interpretation explicit. section 2.4.1

How Deep Learning with R, Third Edition covers it

Cover of Deep Learning with R, Third Edition

liveBook manning.com

Read Deep Learning with R, Third Edition on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Geometry for ProgrammersA derivative measures the rate of change of a function with respect to an input variable.
liveBook manning.com

Grokking AI Algorithms, Second EditionA derivative is a mathematical quantity that measures the instantaneous rate of change of a function with respect to its input.
liveBook manning.com

Deep Learning with Python, Third EditionThe mathematical process of computing derivatives, used here to support gradient-based learning.
liveBook manning.com

Deep Learning with PyTorch, Second EditionA quantity describing how a function changes with respect to an input or parameter.
liveBook manning.com

Advanced Algorithms and Data StructuresA derivative measures the instantaneous rate at which a function changes with respect to its input.

Compare how 6 books explain derivative →

Related concepts

differentiationproduced by · Differentiation is the mathematical operation of determining a function’s rate of change with respect to its inputs.
gradient descentused by · Gradient descent is an iterative optimization method that updates parameters opposite the gradient of an objective function to reduce its value.
slopemeasures · Slope is the rate at which a dependent quantity changes relative to an independent quantity, often represented by a line’s gradient or a coefficient in a linear relationship.
backpropagationused by · Backpropagation is a procedure for computing derivatives of an output or loss with respect to inputs and parameters by applying the chain rule backward through a computational graph.
gradientincludes · A gradient is the vector of partial derivatives of a scalar function with respect to its variables, indicating its local rate and direction of change.
second derivativeincludes · A second derivative is the derivative of a function’s first derivative with respect to the same variable, describing how its rate of change varies.

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with R, Third Edition. The section links point to the parts of the book it draws on.