When an AI finishes your sentence, answers a question, or writes code, it is not starting with a complete response hidden inside it. It repeatedly makes a smaller choice: given everything so far, what token is most likely to come next? A token is a piece of text the system processes. After choosing one token, it adds that token to the sequence and makes the same kind of prediction again. Repeating this simple loop can produce a sentence, a paragraph, or a longer answer. section 2.1
Generation is an iterative loop: predict one next word or token, add it to the context, then repeat. section 2.1
Context is what makes the prediction useful. After “Once upon a…,” “time” is a likely continuation. After “def function(”, code is likely to follow. After “Dear Sir or Madam,” a formal tone is likely. The same prediction mechanism can therefore continue a story, draft an email, or generate code because the preceding tokens make different continuations more or less likely. section 1.2.2 section 2.1
A training turn
Suppose the sequence is: “The performance was unbelievable — the crowd couldn’t …” The expected next token is likely “believe.” During training, the model predicts a continuation, compares it with the actual next token, measures the error, and slightly adjusts its internal parameters. After enormous numbers of such attempts across text and code, it becomes better at recognizing which patterns tend to follow others. Grammar, tone, common phrasing, and programming syntax are learned indirectly through this prediction task rather than entered as explicit rules. section 2.2.1
Likely is not the same as correct. A continuation can sound fluent, relevant, and confident because it fits familiar patterns while still being unreliable. This is why a useful response should not automatically be treated as a verified one. Think of next-token prediction as a model for producing plausible continuations from context, not as a guarantee of factual truth or human-like understanding. section 1.2.2 section 2.1
A practical way to read an AI response is to separate the loop from the impression it creates:
- The prompt and prior output provide context for the next prediction. section 1.2.2
- Each selected token becomes part of the context for the following prediction. section 2.1
- A long, structured answer emerges from many local predictions, not from a single all-at-once decision. section 2.1
- Fluency reflects learned statistical patterns and does not by itself establish accuracy. section 2.1
Next-token prediction began as a language-modeling task, but large-scale transformer-based learning has helped capabilities extend beyond text-only systems. Modern systems can connect text with images and documents, showing how a prediction-centered foundation can support broader forms of input and output. The core takeaway remains simple: outputs are built step by step from patterns in context. section 1.3.2 section 1.2.2












