A machine learning paradigm is a way of framing a task and organizing training data. Common paradigms differ in how they set up the problem, including supervised, self-supervised, reinforcement, and unsupervised learning. Each one tells the model what counts as input, what it should learn from, and how training examples are arranged. section 1.27 section 1.27
The main paradigm here is supervised learning, which uses labeled examples to teach the exact task.
- Pairs each input with a corresponding label, the true output the model should produce. section 1.27
- Uses all supplied examples as training data, often in large numbers. section 1.27
- Depends on manually labeled data in many cases, which can become a costly bottleneck. section 1.27
- Aims for generalization, but can overfit by memorizing training examples instead of learning the process. section 1.27
Next-token training
A sentence can be split into an input and a target label, such as “Better safe than” / “sorry” or “The Eiffel” / “Tower.” That turns ordinary text into supervised training examples for next-token prediction. section 1.27
