A language model learns patterns in text so it can predict what comes next. Given a prompt, it assigns probabilities to possible next words, chooses one, then repeats with the growing sequence. Training compresses patterns observed across many passages into a function that can continue unfamiliar text—not a guarantee that the continuation is true or sensible. section 10.2
A large language model (LLM) scales this approach with a very large transformer and extensive training text. It can produce fluent answers, essays, poems, and conversation. Fluent output is still an imitation of plausible language, not proof of reasoning or reliable knowledge. Training the largest models also demands specialized hardware and substantial compute. section 10.1 section 10.0
To work with a language model, separate the data, model, and application:
- Prepare text as token IDs and retain a vocabulary that maps IDs back to words. Batch long passages to make training more efficient; WikiText2 provides a concrete word-level dataset. section 8.4.1
- Train a word-level recurrent model, then give it a prompt and inspect the generated continuation. Masked <unk> tokens in the benchmark data can interfere with learning and limit accuracy. section 8.5
- Place generation within an application pipeline when needed: parsing and analysis can inform candidate responses, while decision logic and stored conversation history help select them. section 1.8
A plausible answer can contradict itself
Asked how many legs two cows and two bulls have, a Llama 2 example answers “16 legs” but ends its explanation with “4 + 4 = 8 legs.” The response illustrates why even a well-formed answer needs checking before use. section 10.1.1
Figure 10.2 Stochastic chameleons deciding words one at a time
The generation diagram shows candidate words ranked by probability at successive steps. It is a simplified view: each new choice also depends on the words already generated. section 10.2
For practical use, test outputs against the task rather than trusting their tone. Generated text and code may contain errors, misinformation, or bias; review and debugging remain part of the workflow. section 10.0
