Preface
Probabilistic programming is an exciting new field that is quickly gathering interest, moving out of the academic arena and into the world of programmers. In essence, probabilistic programming is a new way of creating models for probabilistic reasoning, which lets you predict or infer things you don’t know from observations. Probabilistic reasoning has long been one of the core approaches to machine learning, where you use a probabilistic model to describe what you know and learn from experience. Before probabilistic programming, probabilistic reasoning systems were limited to models with simple and fixed structures like Bayesian networks. Probabilistic programming sets probabilistic reasoning systems free from these shackles by providing the full power of programming languages to represent models. It’s analogous to moving from circuits to high-level programming languages.
Although I didn’t realize it at the time, I’ve been working on probabilistic programming since my teens, when I developed a soccer simulation in BASIC. The simulation used instructions like “GOTO 1730 + RANDOM * 5” to express random sequencing of events. After careful tuning, the simulation was realistic enough to keep me entertained for hours. Of course, in the intervening years, probabilistic programming has matured a long way from GOTO statements with random targets.