Chain rule
The chain rule is a calculus rule for differentiating a composite function by multiplying derivatives along its composition.
Grokking AI Algorithms, Second Edition
The chain rule is a calculus rule for differentiating a composition of functions by multiplying the derivatives of its successive functions.
- How it is used
- Calculates how individual neural-network weights affect prediction cost during backpropagation and training.
- How deep it goes
- Works through backward output-error and weight-gradient calculations, then connects them to a Python training loop without a theoretical proof.
- Examples
- Uses an artificial neural network’s layer-by-layer weight updates, a sigmoid derivative, and a Python forward/backpropagation training loop.
- What you can do after
- Apply it via a Python neural-network implementation to calculate weight changes from prediction error during backpropagation.
Read how Grokking AI Algorithms, Second Edition explains chain rule →
Deep Learning with R, Third Edition
The chain rule is a calculus rule for differentiating a composition of functions by multiplying derivatives of the component functions.
- How it is used
- Explains how neural-network training obtains parameter gradients from chained tensor operations.
- How deep it goes
- Works from function composition through computation graphs and backward passes, without developing a formal calculus derivation.
- Examples
- Uses a two-layer network, chained tensor operations, and MNIST digit classification with Keras training.
- What you can do after
- Evaluate how backpropagation derives gradients and supports weight updates during neural-network training.
Read how Deep Learning with R, Third Edition explains chain rule →
Deep Learning with Python, Third Edition
The differentiation rule for composing functions, used to compute gradients through layered neural-network operations.
- How it is used
- Explains how backpropagation computes gradients through layered neural-network operations.
- How deep it goes
- Derives the composition identity and applies it to computation graphs, stopping before detailed backward-pass calculations.
- Examples
- Uses composed functions and a two-layer network containing matrix multiplication, ReLU, softmax, addition, and loss.
- What you can do after
- Apply it via computation graphs to calculate gradients through complex chains of differentiable operations.
Read how Deep Learning with Python, Third Edition explains chain rule →
Deep Learning with PyTorch, Second Edition
A differentiation rule for computing the derivative of a composite function by multiplying derivatives along the composition.
- How it is used
- Connects model parameters to loss changes during the backward pass used for learning.
- How deep it goes
- Works through analytical derivatives for a linear model and squared-error loss, then extends the idea to differentiable deep compositions.
- Examples
- Uses a linear model, mean-squared-error loss, manual derivative functions, and a combined gradient function.
- What you can do after
- Apply it to compute parameter gradients manually and understand how automatic differentiation performs the same backward propagation.
Read how Deep Learning with PyTorch, Second Edition explains chain rule →
Related concepts
- backpropagationused by · Backpropagation is a procedure for computing derivatives of an output or loss with respect to inputs and parameters by applying the chain rule backward through a computational graph.
- derivativeproduces · A derivative is a mathematical quantity that measures the instantaneous rate of change of a function with respect to an input variable.
- automatic differentiationused by · Automatic differentiation is a computational technique that evaluates derivatives of a program by systematically applying the chain rule to its elementary operations.
- forward-mode autodiffused by · An automatic differentiation mode that propagates derivatives forward from inputs to outputs and is efficient when there are few inputs relative to outputs.
- gradientuses · A gradient is a vector or tensor of partial derivatives that describes how a function changes with respect to its input variables or parameters.
- Jacobianuses · The matrix of all first-order partial derivatives of a vector-valued function with respect to its inputs.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.