1 Introduction to Bayesian statistics: Representing our knowledge and uncertainty with probabilities
This chapter covers
- Why we need probability
- Comparisons to frequentist statistics
- Large language models as the state-of-the-art application
Welcome to Grokking Bayes! I’m excited that you have chosen to take this journey with me. Bayesian statistics is a way to make predictions and decisions when you’re uncertain about the world. It gives you a mathematical language for expressing your current state of knowledge, updating that knowledge when you see new evidence, and reasoning about what actions to make. Whether you’re deciding if you need to bring an umbrella, estimating the risk of a patient having a disease, or training a machine learning model to generate text, Bayesian statistics provides the tools to handle uncertainty in a structured, quantitative way.
Bayesian statistics treats these unknowns as variables that come with a probability distribution. This means you can formally express your prior knowledge—or lack of knowledge—about a problem and let the data update your beliefs. The output is a full probability distribution that tells you not just a single answer, but how confident you should be in that answer.