A neural network can guess what is in a picture and still get it wrong. Backpropagation is the training technique used to adjust the network’s weights after its guess is compared with the correct answer. Those weights determine how strongly different inputs and features influence the result. section 3.3
Recognizing a digit
Suppose a network sees a picture of a 7 but guesses 1. The mismatch signals an error. During training, backpropagation adjusts the weights so the network can better recognize patterns in later pictures. Providing more labeled pictures gives it more opportunities to learn; a wrong guess is feedback, not proof that the next guess will be correct. section 3.3
Figure 3.3 A backpropagation model with three layers
The three-layer model helps locate backpropagation within a network that processes an input through layers before producing a prediction. section 3.3
Backpropagation also underpins contemporary deep learning. A deep network differs primarily by having more hidden layers, sometimes with different connections between nodes. These layers can build complex features from simpler ones, but training commonly depends on many labeled samples. The practical takeaway is to treat backpropagation as a way to improve predictions from examples, not as a guarantee of correct answers. section 3.4 section 3.3
