Supervised learning learns a mapping from inputs to desired outputs. Given a training dataset D = {(x₁, y₁), …, (xₙ, yₙ)}, it fits a function f that predicts an output y from an input x. Each y is a label: the supervisory signal specifying the desired answer for its paired input. section 1.1
Training adjusts the model’s learnable parameters θ to reduce an objective function that compares predictions with labels. Testing applies the learned mapping to new inputs to produce predictions. The distinction matters: training uses known answers to learn rules; testing uses those rules without a supplied answer. section 1.0 section 1.1
The type of output determines the task:
- Classification predicts a discrete label, such as one of K classes. One possible loss is the average misclassification rate: L(θ) = (1/n) Σᵢ 1[f(xᵢ; θ) ≠ yᵢ]. The indicator is 1 for a wrong prediction and 0 otherwise; minimizing L seeks parameters that make fewer training predictions wrong. section 1.1
- Regression predicts a continuous value, such as a stock price. Its prediction is likewise compared with the desired output, but classification’s misclassification count is not the measure specified for this case. section 1.1
Figure 1.1 Supervised learning: training (left) and testing (right)
The training–testing diagram separates learning from labeled examples from predicting for new inputs. section 1.0
Classification applications
Email spam detection and document classification are settings for supervised classification. Algorithms developed for such tasks include the perceptron, support vector machine, stochastic-gradient-descent logistic regression, Bernoulli naive Bayes, and decision tree (CART). section 5.0
