Chain rule
The chain rule is a calculus rule for differentiating a composite function by combining the derivatives of its component functions.
Grokking AI Algorithms, Second Edition
The chain rule is a calculus rule for differentiating a composition of functions by multiplying the derivatives of its successive functions.
- How it is used
- Uses the chain rule to connect network weights to prediction cost during training.
- How deep it goes
- Explains composite-function derivatives and traces a backward pass from output error to weight gradients.
- Examples
- Works through neural-network weight updates and provides a Python training loop with sigmoid activation.
- What you can do after
- Build a simple neural-network training loop that uses backpropagation to update weights.
Read how Grokking AI Algorithms, Second Edition explains chain rule →
Deep Learning with R, Third Edition
The chain rule is a calculus rule for differentiating a composition of functions by multiplying derivatives of the component functions.
- How it is used
- Explains how neural-network training obtains parameter gradients from chained tensor operations.
- How deep it goes
- Works from function composition through computation graphs and backward passes, without developing a formal calculus derivation.
- Examples
- Uses a two-layer network, chained tensor operations, and MNIST digit classification with Keras training.
- What you can do after
- Evaluate how backpropagation derives gradients and supports weight updates during neural-network training.
Read how Deep Learning with R, Third Edition explains chain rule →
Deep Learning with Python, Third Edition
The differentiation rule for composing functions, used to compute gradients through layered neural-network operations.
- How it is used
- Explains how backpropagation computes gradients through layered neural-network operations.
- How deep it goes
- Derives the composition identity and applies it to computation graphs, stopping before detailed backward-pass calculations.
- Examples
- Uses composed functions and a two-layer network containing matrix multiplication, ReLU, softmax, addition, and loss.
- What you can do after
- Apply it via computation graphs to calculate gradients through complex chains of differentiable operations.
Read how Deep Learning with Python, Third Edition explains chain rule →
Applied Reinforcement Learning
The chain rule is a calculus rule for differentiating a composition of functions by multiplying derivatives of its successive components.
- How it is used
- Applies the chain rule in backpropagation to assign gradients to network parameters from loss.
- How deep it goes
- Defines derivative chaining through composed layers and notes reuse of intermediate results, without a full derivation.
- Examples
- Uses a layered neural network of linear transformations and nonlinearities as the composed-function example.
- What you can do after
- Understand how backpropagation traces loss gradients through layers to estimate each parameter’s contribution to prediction error.
Read how Applied Reinforcement Learning explains chain rule →
Deep Learning with PyTorch, Second Edition
A differentiation rule for computing the derivative of a composite function by multiplying derivatives along the composition.
- How it is used
- Connects model parameters to loss changes during the backward pass used for learning.
- How deep it goes
- Works through analytical derivatives for a linear model and squared-error loss, then extends the idea to differentiable deep compositions.
- Examples
- Uses a linear model, mean-squared-error loss, manual derivative functions, and a combined gradient function.
- What you can do after
- Apply it to compute parameter gradients manually and understand how automatic differentiation performs the same backward propagation.
Read how Deep Learning with PyTorch, Second Edition explains chain rule →
CUDA for LLMs
The chain rule computes the derivative of a composition of functions by multiplying the derivatives of its constituent functions.
- How it is used
- Connects local sensitivities to propagate loss gradients backward through neural-network layers and determine weight effects.
- How deep it goes
- Traces gradients through a simple computational graph from the loss back to its inputs.
- Examples
- Uses linear slopes, partial derivatives, and a multiply-add-square graph with numerical gradients.
- What you can do after
- Understand how to propagate gradients through a calculation graph to find input sensitivities.
Timeless Algorithms, The Foundational Ideas
The chain rule is a calculus rule for differentiating a composite function by multiplying the derivatives of its component functions.
- How it is used
- Supports backpropagation by tracing how network parameters influence the loss through intermediate calculations.
- How deep it goes
- Explains how local effects combine across connected operations, without giving a formal equation.
- Examples
- Traces a network weight’s effects through a node, later layers, the prediction, and the loss.
- What you can do after
- Apply it to combine local effects and measure how a parameter influences the loss.
Read how Timeless Algorithms, The Foundational Ideas explains chain rule →
Also covered in
What to know first
- objective functionAn objective function is a mathematical function that assigns values to candidate solutions or parameter settings, which an optimization procedure seeks to minimize or maximize.
- gradientA gradient is the vector of partial derivatives of a scalar function with respect to its variables, indicating its local rate and direction of change.
- parameterA parameter is a value or named input that specifies or influences the behavior, computation, or output of a function, model, or system.
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
Related concepts
- backpropagationused by · Backpropagation is reverse-mode differentiation that propagates derivatives backward through a computational graph to compute gradients of an output or loss with respect to inputs and parameters.
- derivativeproduces · A derivative is a mathematical quantity that measures the instantaneous rate of change of a function with respect to an input variable.
- automatic differentiationused by · Automatic differentiation is a computational technique that evaluates derivatives of a program by systematically applying the chain rule to its elementary operations.
- forward-mode autodiffused by · An automatic differentiation mode that propagates derivatives forward from inputs to outputs and is efficient when there are few inputs relative to outputs.
- gradientuses · A gradient is the vector of partial derivatives of a scalar function with respect to its variables, indicating its local rate and direction of change.
Questions about chain rule
What is chain rule?
The chain rule is a calculus rule for differentiating a composite function by combining the derivatives of its component functions.
Which Manning books explain chain rule?
Grokking AI Algorithms, Second Edition, Deep Learning with R, Third Edition, Deep Learning with Python, Third Edition, Applied Reinforcement Learning, Deep Learning with PyTorch, Second Edition, CUDA for LLMs and Timeless Algorithms, The Foundational Ideas.
What should I know before learning chain rule?
Manning books that teach chain rule usually introduce objective function, gradient, parameter and language model first.
What is chain rule part of?
Chain rule is part of calculus. Calculus is the branch of mathematics that studies limits, continuous change, derivatives, and integrals.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.






