13 Bayesian experimental design: Data collection the smart way
This chapter covers
- Entropy as a measure of uncertainty
- Minimizing entropy to gain information
- The expected information gain criterion for Bayesian experimental design
In most of the problems we’ve seen so far, the data were already there. We observed it, modeled it, and used the updated model to make predictions. At the heart of many of these problems is an unknown random variable of interest that we don’t get to observe: a patient’s true condition, a system’s underlying state, a parameter that governs how the world works. The data we observe are only a window into that hidden quantity, yielding partial information about what the value of that quantity could be.
In many real-world settings, we don’t just passively look through that window—we get to choose it.
- A doctor can order specific tests out of a wide range of tests.
- A scientist can run different experiments.
- A detective can decide whom to interview to look for clues.
Each choice reveals different information about the same unknown, and some are more informative than others. So, the question becomes: what data should we collect to learn about what we care about? This is the central idea behind Bayesian experimental design: choosing observations—that is, designing experiments—not just to get data for the sake of data, but to actively learn about unknown quantities of interest.
