liveBook · Manning

Backpropagation

As explained in Optimization Algorithms: AI techniques for design, planning, and control problems

Backpropagation is an algorithm that computes derivatives of a neural-network loss with respect to its parameters by propagating error information backward through the network.

Backpropagation computes how the loss changes with respect to each neural-network weight and bias by propagating error information backward from the output layer. Those derivatives guide weight updates, commonly through gradient descent, and repeated training reduces prediction error. section 9.7

A supervised training cycle proceeds as follows:

  • Initialize weights and biases with small random values. section 9.7
  • Feed inputs forward through weighted layers and activation functions. section 9.7
  • Compare predictions with labeled targets using a loss function. section 9.7
  • Propagate the error backward, compute parameter derivatives, and update weights in the direction that reduces loss. section 9.7

Penguin classification

A two-layer network classifies penguin species from bill measurements and body mass. The forward computation uses four inputs, ten hidden units, and three output classes; the complete parameter vector has 83 values. The demonstrated implementation evaluates candidate parameters with particle swarm optimization rather than applying backpropagation updates.

How Optimization Algorithms: AI techniques for design, planning, and control problems covers it

How deep it goes
Provides a procedural overview from initialization through updates but does not derive chain-rule equations or implement gradient calculations.
Examples
Uses a two-layer penguin-species classifier with four features, ten hidden units, three classes, and an 83-parameter representation.
What it assumes
Assumes basic neural-network components such as weights, biases, layers, activations, outputs, and loss functions.
What you can do after
Evaluate/understand the role of backward derivatives in neural-network training and distinguish them from PSO parameter search.
How it connects
Connects feedforward computation, activation functions, loss functions, gradient descent, and supervised learning.
Cover of Optimization Algorithms: AI techniques for design, planning, and control problems
liveBookmanning.com
Read Optimization Algorithms: AI techniques for design, planning, and control problems on liveBook Get the book at manning.com

Also explained in

Compare how 17 books explain backpropagation →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Optimization Algorithms: AI techniques for design, planning, and control problems. The section links point to the parts of the book it draws on.