Recognizing an image requires a way to turn raw input into useful features. Deep learning addresses this problem with neural networks that learn representations through multiple layers, rather than relying entirely on features specified in advance. Successive transformations can reveal patterns that would be difficult to handcraft. section 11.0 section 11.1
A layer can be written as hₗ = σ(Wₗhₗ₋₁ + bₗ), where hₗ₋₁ is the previous layer’s output, Wₗ contains weights, bₗ contains biases, and σ is an activation function. The weights and biases parameterize the transformation; the activation adds nonlinearity. Depth determines how many transformations are composed, while width determines how many patterns a layer can represent at one stage. section 11.8 section 11.0
Training repeatedly connects prediction to correction. A forward pass produces an output; a loss function measures its discrepancy from the observed outcome. Backpropagation applies the chain rule in reverse to calculate each parameter’s influence on the loss. Gradient descent then updates the parameters using those gradients and a chosen learning rate. As parameters change, the intermediate representations change too. section 10.5.1 section 11.8
Prediction and decision
A classification network can produce a probability, which a predetermined threshold converts into a class or operational response. These are distinct from the probability itself. A regression network instead produces a continuous numerical prediction. Training evaluates the continuous output: cross-entropy is used for classification, while squared error is widely used for regression. section 11.8
The modern rise of deep learning followed decades of uneven progress. Larger data sets, greater computing power—including graphics processing units—and improved network design and training made multilayer networks practical at scale. The 2015 synthesis by Yann LeCun, Yoshua Bengio, and Geoffrey Hinton captured this shift toward learned representations. section 11.0 section 11.1
Learning a low training loss is not the same as generalizing to unseen observations. What a network can learn is constrained by its architecture, data and computation. Its training also brings together likelihood, cross-entropy and probabilistic reasoning, linking representation learning to broader questions of prediction and uncertainty. section 10.5.1



















