chapter one

1 Introduction to Bayesian statistics: Representing our knowledge and uncertainty with probabilities

 

This chapter covers

  • Representing knowledge and quantifying uncertainty with probability
  • Comparing Bayesian statistics to frequentist statistics
  • Large language models as a motivating example

Welcome to Grokking Bayes! I’m excited that you have chosen to take this journey with me. Bayesian statistics is a way to make predictions and decisions when you’re uncertain about the exact state of the world. It gives you a mathematical language for expressing your current state of knowledge, updating that knowledge when you see new evidence, and reasoning about what actions to make. Whether you’re deciding if you need to bring an umbrella, estimating the risk of a patient having a disease, or training a machine learning model to generate text, Bayesian statistics provides the tools to handle uncertainty in a structured, quantitative way.

While sharing the same goals of analyzing and interpreting data as other approaches in statistics, Bayesian methods treat these unknowns as variables that come with a measure of uncertainty. With this formulation, you can formally express your prior knowledge—or lack of knowledge—about a problem and let the data update your beliefs. The output is a full probability distribution that tells you not just a single answer, but how confident you should be in that answer.

Why uncertainty matters

Different ways to build a weather app

Levels of granularity of reasoning

Introducing Bayesian probability

Representing our belief about an unknown object

Moving from binary to categorical

Bayesian versus frequentist probability

Large language models and Bayesian probability

What you will learn in this book

Summary