Language modeling predicts what word or word sequence is likely to come next after a preceding sequence. It assigns probabilities to candidate continuations: after “French is a beautiful …,” a trained model should give “language” a higher probability than “country.” section 5.2.1
A language model learns from a text corpus by observing which words occur together and which do not. From these patterns it forms probabilistic rules for continuations in different contexts. Training on very large volumes of text produces a large language model (LLM), whose writing capabilities can support conversation and other text tasks. section 5.2.1 section 5.0
Language modeling provides the underlying prediction capability behind several LLM uses:
- Generate human-like text by repeatedly predicting a next word from preceding words. section 5.0 section 5.2.1
- Support translation, summarization, question answering, and sentiment analysis. section 5.0
- Bring broad text knowledge to phishing detection, alongside a progression from rules and classical machine learning to deep learning. section 5.0
Choosing the next word
Given the prefix “It will,” a model scores possible next words with a probability distribution. A simple greedy approach selects the most likely candidate; repeating that choice extends the generated text one word at a time. section 5.2.1
Figure 5.7 Toy example of how an LLM predicts the next word (given the preceding words) by sampling from a distribution of likely words. In this example, the LLM always chooses the most likely next word – called the greedy search approach.
This toy prediction sequence makes the probability distribution and greedy next-word selection concrete. section 5.2.1






