Derivative
A derivative is a mathematical quantity that measures the instantaneous rate of change of a function with respect to an input variable.
Geometry for Programmers
A derivative measures the rate of change of a function with respect to an input variable.
- How it is used
- Uses derivatives to quantify curve growth, tangent slopes, smooth joins, and curvature for geometric programming.
- How deep it goes
- Explains tangent-line intuition, derivative functions, standard rules, higher derivatives, and symbolic computation, while deferring broader calculus applications.
- Examples
- Illustrates linear, quadratic, cubic, sine, and cosine derivatives, plus a cubic polynomial used to locate extrema.
- What you can do after
- Apply it via SymPy to differentiate functions and interpret slopes, extrema, tangent constraints, and curve smoothness.
Inside Deep Learning: Math, Algorithms, Models
The rate of change of a function with respect to an input variable.
- How it is used
- Uses derivatives to guide minimization of functions and neural-network losses.
- How deep it goes
- Works through a quadratic derivative and PyTorch updates, while noting that complicated functions may lack an exact solution.
- Examples
- Uses f(x) = (x − 2)² and a PyTorch loop to illustrate derivatives and minimization.
- What you can do after
- Apply PyTorch automatic differentiation and gradient descent to minimize a function.
Read how Inside Deep Learning: Math, Algorithms, Models explains derivative →
Grokking AI Algorithms, Second Edition
A derivative is a mathematical quantity that measures the instantaneous rate of change of a function with respect to its input.
- How it is used
- Uses derivatives to guide weight adjustments during neural-network training.
- How deep it goes
- Explains slope intuition and the chain rule’s role in a backward pass, with a sigmoid derivative implementation.
- Examples
- Illustrates derivatives with a dark hillside, velocity and acceleration, and a Python sigmoid function.
- What you can do after
- Apply it in Python to calculate a sigmoid derivative for neural-network training.
Read how Grokking AI Algorithms, Second Edition explains derivative →
Deep Learning with R, Third Edition
A derivative is a measure of how a function's output changes in response to an infinitesimal change in an input.
- How it is used
- Explains how optimization uses local output changes to update neural-network parameters and reduce loss.
- How deep it goes
- Defines local slope, extends derivatives to tensor-valued functions as gradients, and applies them procedurally in mini-batch optimization.
- Examples
- Uses cosine, linear functions, model-weight loss, forward and backward passes, and random mini-batches as examples.
- What you can do after
- Apply it via gradient descent to choose parameter updates that reduce a neural network’s loss.
Read how Deep Learning with R, Third Edition explains derivative →
Deep Learning with Python, Third Edition
The mathematical process of computing derivatives, used here to support gradient-based learning.
- How it is used
- Supports optimization by indicating how inputs or model weights should change to reduce a function’s value.
- How deep it goes
- Defines local slope, linear approximation, and tensor gradients, while staying intuitive and avoiding heavy notation or derivations.
- Examples
- Uses cosine and linear functions, local slope diagrams, and a weight-to-loss computation involving predictions and targets.
- What you can do after
- Evaluate/understand how derivatives and gradients guide parameter updates for minimizing loss.
Read how Deep Learning with Python, Third Edition explains derivative →
Deep Learning with PyTorch, Second Edition
A quantity describing how a function changes with respect to an input or parameter.
- How it is used
- Uses analytical derivatives to determine how model loss changes and support parameter fitting in PyTorch.
- How deep it goes
- Derives loss and linear-model derivatives with the chain rule, then assembles them into a two-parameter gradient.
- Examples
- Demonstrates derivatives with a linear model, mean squared loss, weight and bias parameters, and PyTorch-style gradient functions.
- What you can do after
- Apply it to calculate a model’s gradient analytically instead of estimating change through repeated finite-difference evaluations.
Read how Deep Learning with PyTorch, Second Edition explains derivative →
Advanced Algorithms and Data Structures
A derivative measures the instantaneous rate at which a function changes with respect to its input.
- How it is used
- Optimizes the number of Bloom-filter hash functions by locating a function's minimum.
- How deep it goes
- Develops first- and second-order derivative use, but skips the second-order calculation details.
- Examples
- Uses the false-positive-rate function and the optimal hash-function count k as its worked application.
- What you can do after
- Apply it via calculus to choose an approximately optimal Bloom-filter hash-function count.
Read how Advanced Algorithms and Data Structures explains derivative →
Also covered in
- Machines that ThinkIn early access (MEAP)
- Deep Learning with JAX
- Causal AI
What to know first
- objective functionAn objective function is a mathematical function that assigns values to candidate solutions or parameter settings, which an optimization procedure seeks to minimize or maximize.
- parameterA parameter is a value or named input that specifies or influences the behavior, computation, or output of a function, model, or system.
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- scalarA scalar is a single numerical quantity with no vector or multidimensional components.
Related concepts
- differentiationproduced by · Differentiation is the mathematical operation of determining a function’s rate of change with respect to its inputs.
- gradient descentused by · Gradient descent is an iterative optimization method that updates parameters in the direction opposite an objective function’s gradient to reduce the objective’s value.
- slopemeasures · Slope is the rate at which one quantity changes with respect to another, represented for a straight line by the ratio of vertical change to horizontal change.
- backpropagationused by · Backpropagation is reverse-mode differentiation that propagates derivatives backward through a computational graph to compute gradients of an output or loss with respect to inputs and parameters.
- gradientincludes · A gradient is the vector of partial derivatives of a scalar function with respect to its variables, indicating its local rate and direction of change.
- chain ruleproduced by · The chain rule is a calculus rule for differentiating a composite function by combining the derivatives of its component functions.
Questions about derivative
What is derivative?
A derivative is a mathematical quantity that measures the instantaneous rate of change of a function with respect to an input variable.
Which Manning books explain derivative?
Geometry for Programmers, Inside Deep Learning: Math, Algorithms, Models, Grokking AI Algorithms, Second Edition, Deep Learning with R, Third Edition, Deep Learning with Python, Third Edition, Deep Learning with PyTorch, Second Edition and Advanced Algorithms and Data Structures.
What should I know before learning derivative?
Manning books that teach derivative usually introduce objective function, parameter, language model and scalar first.
What kinds of derivative are there?
Kinds of derivative include gradient.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.






