Probability describes how strongly a language model favors token sequences given their preceding context. Training rewards predicting continuations that resemble its data, so probability reflects learned patterns—not whether a claim matches external facts. section 6.1.2
When context strongly matches learned patterns, outputs often align with known information. With weak or ambiguous matches, several continuations may compete; the model may still choose the likeliest rather than signal uncertainty, producing plausible but unsupported claims. section 6.1.2
Figure 6.1 Models learn patterns from training data and extend them to new cases. In this example, animals associated with water are often labeled as fish. When no clear example is available, as with the dolphin, the model does not stop or admit uncertainty. It continues by extending the strongest available pattern, incorrectly classifying the dolphin as a fish rather than a mammal.

The dolphin example illustrates how a model can extend a strong pattern—associating water animals with fish—even when that pattern leads to an incorrect answer. section 6.1.2





















