Think of GPT as an intelligent autocomplete: it generates text by predicting the most likely next word, producing coherent continuations rather than following hand-written rules. section 1.5
Understand GPT within the larger learning process:
- GPT begins as an unsupervised model that learns language patterns from data and completes sentences. section 1.5
- Fine-tuning then uses supervised and reinforcement learning to improve the initial output for human conversation. section 1.5
From completion to conversation
Given the beginning of a sentence, GPT selects a likely continuation. Further human refinement helps turn this completion ability into the conversational behavior associated with ChatGPT. section 1.5
