Probability measures how likely a possible outcome is. In generative AI, it describes how models use learned patterns to choose what content is most likely to come next, rather than retrieving a guaranteed stored answer. section 1.3 section 1.5.1
Probability shapes generative AI behavior in several practical ways:
- Training exposes a model to large collections of text or source code, allowing it to learn statistical patterns. section 1.3 section 1.5.1
- Generation predicts the most probable next word or character, then repeats the process using the new output as additional context. section 1.5.1
- Prediction makes outputs variable, permits unfamiliar combinations, and can produce confident errors, so generated code and claims require verification. section 1.5.1 section 1.12
From dog images to generated code
A model shown thousands of dog photographs can learn patterns such as body shape, legs, and tails, assigning probabilities to possible features. A coding model similarly learns patterns from source code and assembles a likely new response. The result may be useful, but likelihood is not correctness. section 1.3 section 1.12
Figure 1.1 Integrated tools use a sophisticated system to generate code. The process starts with your prompt, and the assistant gathers up documentation and source code to see whether your answer can come from these sources. It makes a best guess at what you’re looking for and generates a response. Your acceptance of these responses helps train the assistant in the future (unless you’ve blocked feedback).
Trace how a prompt, documentation, and source code contribute to an assistant’s probabilistic best guess. section 1.3
