Next-token prediction trains a language model to predict the token that follows a sequence of preceding tokens. Its performance is measured by how accurately it predicts that next token. This objective rewards alignment with patterns in training data, not direct verification of factual truth. section 6.1.2
During generation, a decoder-only model uses the current context to make some continuations more likely than others, then produces text token by token. Tasks such as summarization and translation can be framed this way, allowing one model structure to serve different purposes. section 4.1.8 section 6.4.1
The model’s learned parameters shape its predictions, but they are not a database from which facts are retrieved. At inference time, a fact can influence an answer only if it is encoded in those parameters or present in the active input. Earlier conversation turns, for example, must be reintroduced into the context to affect generation. section 6.4.1
Plausible continuation is not the same as factual correctness. The objective does not require a model to check external facts or express uncertainty, so weak or ambiguous patterns can produce confident hallucinations. Evaluation and fine-tuning practices that reward answers more than refusals can reinforce this tendency. section 6.1.2
Figure 6.1 Models learn patterns from training data and extend them to new cases. In this example, animals associated with water are often labeled as fish. When no clear example is available, as with the dolphin, the model does not stop or admit uncertainty. It continues by extending the strongest available pattern, incorrectly classifying the dolphin as a fish rather than a mammal.

An animal-classification example illustrates how extending a familiar pattern can produce a plausible but incorrect answer. section 6.1.2
From prediction to mathematics
A study using MathCAMPS tested models at different training stages. Models trained only through next-token prediction learned mathematical skills in an order correlated with a human-designed curriculum. The result suggests that prediction can support mathematical structure, without showing that models learn as children do. section 9.5.2
Fluent output alone does not establish human-like understanding or intention. At the same time, the ability to reuse learned structure across tasks leaves open a broader debate about what forms of intelligence such systems may exhibit. section 4.1.8 section 9.5.2












