Derivative
A derivative is a mathematical quantity that measures the instantaneous rate of change of a function with respect to an input variable.
Geometry for Programmers
A derivative measures the rate of change of a function with respect to an input variable.
- How it is used
- Uses derivatives to quantify curve growth, tangent slopes, smooth joins, and curvature for geometric programming.
- How deep it goes
- Explains tangent-line intuition, derivative functions, standard rules, higher derivatives, and symbolic computation, while deferring broader calculus applications.
- Examples
- Illustrates linear, quadratic, cubic, sine, and cosine derivatives, plus a cubic polynomial used to locate extrema.
- What you can do after
- Apply it via SymPy to differentiate functions and interpret slopes, extrema, tangent constraints, and curve smoothness.
Grokking AI Algorithms, Second Edition
A derivative is a mathematical quantity that measures the instantaneous rate of change of a function with respect to its input.
- How it is used
- Guides neural-network weight adjustments by measuring how a change in a weight affects cost.
- How deep it goes
- Explains slope intuition, partial-derivative notation, the Chain Rule in backpropagation, and a Python sigmoid-derivative function.
- Examples
- Uses a dark hillside, velocity and acceleration, weight-versus-cost plots, and a Python sigmoid-derivative function.
- What you can do after
- Build it from scratch in a small Python neural-network training loop that uses derivatives to update weights.
Read how Grokking AI Algorithms, Second Edition explains derivative →
Deep Learning with R, Third Edition
A derivative is a measure of how a function's output changes in response to an infinitesimal change in an input.
- How it is used
- Explains how optimization uses local output changes to update neural-network parameters and reduce loss.
- How deep it goes
- Defines local slope, extends derivatives to tensor-valued functions as gradients, and applies them procedurally in mini-batch optimization.
- Examples
- Uses cosine, linear functions, model-weight loss, forward and backward passes, and random mini-batches as examples.
- What you can do after
- Apply it via gradient descent to choose parameter updates that reduce a neural network’s loss.
Read how Deep Learning with R, Third Edition explains derivative →
Deep Learning with Python, Third Edition
The mathematical process of computing derivatives, used here to support gradient-based learning.
- How it is used
- Supports optimization by indicating how inputs or model weights should change to reduce a function’s value.
- How deep it goes
- Defines local slope, linear approximation, and tensor gradients, while staying intuitive and avoiding heavy notation or derivations.
- Examples
- Uses cosine and linear functions, local slope diagrams, and a weight-to-loss computation involving predictions and targets.
- What you can do after
- Evaluate/understand how derivatives and gradients guide parameter updates for minimizing loss.
Read how Deep Learning with Python, Third Edition explains derivative →
Deep Learning with PyTorch, Second Edition
A quantity describing how a function changes with respect to an input or parameter.
- How it is used
- Uses analytical derivatives to determine how model loss changes and support parameter fitting in PyTorch.
- How deep it goes
- Derives loss and linear-model derivatives with the chain rule, then assembles them into a two-parameter gradient.
- Examples
- Demonstrates derivatives with a linear model, mean squared loss, weight and bias parameters, and PyTorch-style gradient functions.
- What you can do after
- Apply it to calculate a model’s gradient analytically instead of estimating change through repeated finite-difference evaluations.
Read how Deep Learning with PyTorch, Second Edition explains derivative →
Advanced Algorithms and Data Structures
A derivative measures the instantaneous rate at which a function changes with respect to its input.
- How it is used
- Optimizes the number of Bloom-filter hash functions by locating a function's minimum.
- How deep it goes
- Develops first- and second-order derivative use, but skips the second-order calculation details.
- Examples
- Uses the false-positive-rate function and the optimal hash-function count k as its worked application.
- What you can do after
- Apply it via calculus to choose an approximately optimal Bloom-filter hash-function count.
Read how Advanced Algorithms and Data Structures explains derivative →
Related concepts
- differentiationproduced by · Differentiation is the mathematical operation of determining a function’s rate of change with respect to its inputs.
- gradient descentused by · Gradient descent is an iterative optimization method that updates parameters opposite the gradient of an objective function to reduce its value.
- slopemeasures · Slope is the rate at which a dependent quantity changes relative to an independent quantity, often represented by a line’s gradient or a coefficient in a linear relationship.
- backpropagationused by · Backpropagation is a procedure for computing derivatives of an output or loss with respect to inputs and parameters by applying the chain rule backward through a computational graph.
- gradientincludes · A gradient is the vector of partial derivatives of a scalar function with respect to its variables, indicating its local rate and direction of change.
- second derivativeincludes · A second derivative is the derivative of a function’s first derivative with respect to the same variable, describing how its rate of change varies.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.