Backpropagation traces a neural network’s error backward through its layers to determine how each weight contributes to the loss. These sensitivities provide the gradients used to improve the network’s predictions. section 3.2.2
A forward pass transforms inputs into an output, and the loss measures how far that output is from the desired answer. Backpropagation then asks how the loss would change if each weight changed slightly. Derivatives express those sensitivities; together, they form the gradient. An optimizer uses the gradient to adjust parameters toward lower loss. section 3.2.2 section 3.6
Modern software frameworks use automatic differentiation to calculate the derivatives needed for backward propagation. Training repeats forward and backward passes with optimization steps, while inference typically uses only a forward pass with fixed parameters. section 3.2.2 section 4.6.1
At large scale, backward passes are part of a substantial engineering workload. Gradients must be aggregated across machines, and intermediate activations must be stored or recomputed within memory limits. section 4.6.1
























