A conventional algorithm follows instructions supplied by its designer: a sorting algorithm, for example, takes a list and returns it in order. A machine-learning algorithm instead learns rules from data. In supervised learning, it compares its predictions with desired outputs during training, then applies the learned rules to new inputs during testing. section 1.0
For supervised learning, write the training data as D = {(x₁, y₁), …, (xₙ, yₙ)}. The goal is to learn a mapping f from an input x to an output y; θ denotes its learnable parameters. A loss function measures how far predictions are from the supplied labels. One classification loss is the average misclassification rate: (1/n) Σᵢ 1[f(xᵢ; θ) ≠ yᵢ]. Training seeks parameters θ that minimize this loss. section 1.1
The main groupings distinguish what is learned and how the task is framed:
- Supervised learning uses input–output pairs. Predicting one of K discrete classes is classification; predicting a continuous value, such as a stock price, is regression. section 1.1
- Unsupervised learning is another category; clusters of data points provide a visual example. section 1.1
- Deep learning models learn representations; backpropagation is identified as the means of minimizing an objective function. section 1.6
Figure 1.1 Supervised learning: training (left) and testing (right)
The training–testing diagram separates learning rules from labeled examples from making predictions on new inputs. section 1.0
Understanding an algorithm from scratch makes its assumptions and shortcomings easier to inspect. That supports choosing an approach, explaining its results, debugging a model, and adapting an algorithm when an existing implementation does not fit the task. section 1.2
