Delta
In neural-network backpropagation, a delta is an error signal representing the loss derivative with respect to a neuron's preactivation.
Classic Computer Science Problems in Swift: Essential techniques for practicing programmers
A delta is a neural-network error signal that combines a neuron's error with the derivative of its activation function to quantify the adjustment direction.
- How it is used
- Directs weight changes during backpropagation by assigning an error signal to each output and hidden neuron.
- How deep it goes
- Works through output- and hidden-layer formulas and Swift methods, but leaves their partial-derivative derivation to other treatments.
- Examples
- Implements separate Swift methods for output- and hidden-layer deltas, followed by a weight-update method.
- What you can do after
- Build it from scratch in Swift by storing neuron deltas, calculating them by layer, and applying them to weight updates.
Math and Architectures of Deep Learning
An intermediate variable used in backpropagation to represent the rate of change of loss with respect to a layer's pre-activation output.
- How it is used
- Uses delta to propagate loss derivatives backward and calculate layer-wise weight and bias gradients.
- How deep it goes
- Defines delta, derives output and hidden-layer equations, and demonstrates a PyTorch implementation for a simple multilayer perceptron.
- Examples
- Demonstrates output-layer and hidden-layer deltas through equations and a PyTorch backward-propagation listing.
- What you can do after
- Build it from scratch in PyTorch by computing deltas and using them to form parameter gradients.
Read how Math and Architectures of Deep Learning explains delta →
Related concepts
- backpropagationused by · Backpropagation is a procedure for computing derivatives of an output or loss with respect to inputs and parameters by applying the chain rule backward through a computational graph.
- set differenceuses · Set difference is the set of elements that belong to one set but not to another.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.