A training dataset is a collected set of input examples used to estimate a model’s parameters. In supervised learning, each input is paired with a desired output, also called a target or ground truth. Training seeks parameter values that make model outputs match these known outputs as closely as possible; the resulting model is then used for inferencing on new inputs. section 1.1 section 1.2
A dataset is not necessarily labeled. For distribution modeling, training data can be only sampled input points X = {x(1), x(2), …, x(n)}. The model then estimates parameters that fit or explain the observed point cloud, rather than predicting supplied target values. A Gaussian model, for example, estimates its mean and covariance from unlabeled samples. section 6.6 section 13.2
The same dataset plays different roles depending on the learning setting:
- In supervised classification or regression, it is a set of ⟨input, ground-truth output⟩ pairs. Labels are commonly produced through manual labeling, annotation, or curation. section 7.1 section 8.0
- In classifier modeling, labeled sample points stand in for an unknown decision boundary; the model must generalize from those examples to a random new scene. section 3.1.3
- In unsupervised density modeling, it is an unlabeled point cloud whose structure guides parameter fitting, such as fitting Gaussian-mixture components. section 6.9.4
Threat-score examples
A small supervised dataset can contain inputs such as hardness = 0.01 and sharpness = 0.02 paired with a desired threat score of −0.90, alongside other input-output pairs. A weighted model produces a prediction for each input; training changes its weights and bias to reduce the difference from the corresponding target. Repeating this over many examples makes the parameters approach values that fit the dataset. section 1.1 section 1.3.5
For a dataset of N + 1 supervised instances, each input x(i) has a target ygt(i). A model produces ypredicted(i), and training aggregates the mismatch across examples, often as squared error. At machine-learning scale, iterative methods can process a few examples at a time instead of retaining the entire dataset in memory. section 1.3.5
Figure 6.12 Progression of maximum likelihood estimation for GMM parameters
|
(a) Training data point cloud (target for fitting) |
|
(b) Fitted GMM’s sample point cloud at step 0 |
|
(c) Fitted GMM’s sample point cloud at step 5 |
|
(d) Fitted GMM’s sample point cloud at step 15. It almost matches the target. |
Compare the target training-data point cloud with GMM sample clouds across fitting steps to see unlabeled data as the geometric target for parameter estimation. section 6.9.4
Dataset adequacy matters because training only constrains a model on its sampled inputs. An architecture with sufficient expressive power, trained with adequate data, may produce accurate outputs on unseen inputs, but inferencing remains probabilistic rather than guaranteed. Labels are valuable but expensive, motivating unsupervised, semi-supervised, and self-supervised approaches. section 7.1 section 1.8
