A large language model (LLM) predicts what word might come next in a sentence. Because several words may fit, it assigns different probabilities to possible continuations: after “Bayesian probability,” “theory” or “examples” are plausible, while “cat” is not. section 1.7
The prediction depends on both the preceding words and the data used to train the model. Conditioning on different starting phrases lets an LLM account for different contexts. Bayesian probability offers one way to relate these conditional probabilities to a prior, data, and posterior; LLMs can be Bayesian in spirit without that making them straightforwardly Bayesian models. section 1.7
This expression represents the probability of a continuation given the phrase “Bayesian probability” and training data D. section 1.7
























































































