liveBook · Manning

Backpropagation

As explained in Fight Fraud with Machine Learning

Backpropagation is an algorithm that computes gradients of a loss with respect to neural-network parameters by propagating error derivatives backward through the network.

Backpropagation trains neural networks by computing how much each parameter contributes to a loss, then using those gradients to update the parameters. It follows a forward pass, loss calculation, backward propagation through the chain rule, and parameter update. section 4.1.1

A typical training iteration proceeds as follows:

PyTorch fake-voice training loop

A training loop clears existing gradients, performs a forward pass, computes the criterion, calls `loss.backward()`, and runs `optimizer.step()`. Validation performs the forward pass and loss calculation without backpropagating the error. section 8.2.2

Figure 4.6 Steps in the training of a deep learning model using gradient descent.

Use the training diagram to connect initialization, forward prediction, loss calculation, backpropagation, and parameter updates. section 4.1.1

How Fight Fraud with Machine Learning covers it

How deep it goes
Explains a complete gradient-descent workflow and chain-rule gradient propagation, but does not derive gradients for specific network layers.
Examples
Uses dog-versus-cat classification, same-versus-different face pairs, and a PyTorch fake-voice classifier.
What it assumes
Assumes labeled examples, model predictions, and a loss function while introducing the training sequence from initialization onward.
What you can do after
Apply it via PyTorch to train parameters and distinguish training behavior from validation evaluation.
How it connects
Connects gradient descent, loss functions, binary cross-entropy, chain rule, optimizers, convolutional neural networks, and Siamese networks.

Also explained in

Compare how 25 books explain backpropagation →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Fight Fraud with Machine Learning. The section links point to the parts of the book it draws on.