liveBook · Manning

Supervised learning

As explained in Fight Fraud with Machine Learning

Supervised learning is machine learning in which a model learns from input examples paired with target outputs or labels.

Supervised learning trains a model on paired inputs and target outputs. For conversational models, the pairing can be a question as input (X) and a manually written answer as output (y). Fine-tuning on these pairs changes a pre-trained GPT model from mainly continuing text into responding to prompts. section 5.2.1

Supervised fine-tuning for responses

After unsupervised pre-training on text, a GPT model is fine-tuned with questions and corresponding answers from manual labelers. The labeled examples teach the model to produce an answer for a question rather than simply predict a continuation. section 5.2.1

Figure 5.12 Demonstration of Supervised Fine-Tuning of a ChatGPT model.

The figure contrasts unlabeled text used for pre-training with X–y question-and-answer pairs used for supervised fine-tuning. section 5.2.1

Supervised fine-tuning is one stage in a broader training pipeline. It follows compute-intensive unsupervised pre-training and precedes reinforcement learning that uses labeler rankings or scores to train a reward model and further tune the model. section 5.2.1

How Fight Fraud with Machine Learning covers it

How deep it goes
Explains paired X–y data and its role between pre-training and reinforcement learning, without describing the fine-tuning algorithm.
Examples
Uses ChatGPT fine-tuning with manual-labeler questions as inputs and corresponding answers as outputs.
What it assumes
Introduces supervised data by contrasting labeled X–y pairs with unlabeled text used in unsupervised learning.
What you can do after
Evaluate/understand how labeled question-answer pairs make a pre-trained GPT model respond to prompts.
How it connects
Connects supervised learning to unsupervised learning, GPT, fine-tuning, reinforcement learning, and reward models.

Also explained in

Compare how 24 books explain supervised learning →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Fight Fraud with Machine Learning. The section links point to the parts of the book it draws on.