Why can a language model produce a convincing continuation without necessarily answering your question? Before it becomes an assistant, it is trained to notice what text is statistically likely to come next. This initial phase is pretraining: exposure to a vast collection of text so the model can learn broad patterns of language. section 2.2 section 2.2.1
Training material can include books, articles, websites, Wikipedia pages, discussions, documentation, and source code. These materials are converted into tokens. Given a sequence of tokens, the model predicts the next one; its prediction is compared with the actual next token, and the resulting error makes small adjustments to internal parameters. Repeating this process billions or trillions of times improves its ability to recognize regularities. section 2.2.1 section 2.2.1
A next-token prediction
Given “The performance was unbelievable — the crowd couldn’t…”, a likely next token is “believe.” Early in training, the model may favor unrelated tokens. With repeated correction, it learns that some words, structures, tones, and styles commonly occur together. It is not given hand-written rules for grammar, sarcasm, storytelling, or programming syntax; those patterns emerge indirectly from prediction. section 2.2.1 section 2.2.1
Figure 2.6 A child and a language model learn language through the same fundamental mechanism: exposure and pattern recognition, without explicit instruction.

The child analogy captures the central intuition: extensive exposure and pattern recognition can produce language-like behavior without explicit instruction in grammar rules. section 2.2.1
Pretraining builds a foundation for later stages, not a fully formed helper. A raw pretrained model treats input as text to continue, so several continuations may fit the same prompt. Fine-tuning teaches it to interpret input as an instruction, alignment shapes behavior using human feedback, and specialization redirects the same foundation toward a domain. Throughout the arc, the core mechanism remains next-token prediction. section 2.2 section 2.2.4 section 2.3
The practical takeaway is modest but useful: fluent output reflects learned statistical patterns, not an inherent intention to help or guaranteed correctness. Knowing that distinction helps explain why a model can sound coherent, offer several plausible continuations, or need additional training to behave like a conversational assistant. section 2.2 section 2.3

















