liveBook · Manning

Decision theory

As explained in Grokking Bayes

Decision theory is the formal study of choosing among actions under uncertainty according to preferences, objectives, or expected consequences.

A prediction does not, by itself, tell you what to do. Suppose a new website design will probably help, but could also cause a costly loss. The choice depends on both how likely each outcome is and how much that outcome matters. Bayesian decision theory combines those beliefs and preferences to choose an action under uncertainty. section 12.0 section 11.10

Start by assigning each possible outcome a utility: a number representing how desirable it is. For each action, multiply every outcome’s utility by its probability, then add the results. This probability-weighted average is the action’s expected utility. Choose the action with the highest expected utility—not necessarily the one with the most likely favorable outcome. section 12.2

Should the new design launch?

Launching yields +100,000 if the design improves results and −20,000 otherwise; not launching yields 0. With a 0.97 probability of improvement, launch has expected utility of 100,000 × 0.97 − 20,000 × 0.03 = 96,400, so launching wins. If the possible loss rises to 4,000,000, the same probabilities give −23,000: now not launching wins. The beliefs stayed the same; the consequences changed. section 12.2 section 11.10

Figure 12.3 After adding up the weighted utilities associated with each action, we obtain the action’s expected utility. The action with the highest expected utility is chosen by Bayesian decision theory.

Follow each outcome branch back to its action: weight its utility by its probability, add the branches, then compare actions. section 12.2

For a sequence of decisions, an action can also teach you something useful for later choices. Maximizing utility across the whole sequence can therefore favor exploration over the best immediate payoff. Searching every possible branch becomes expensive as the sequence grows; upper confidence bound, Thompson sampling, and Monte Carlo tree search offer practical approximations. section 12.9 section 12.10 section 12.11

How Grokking Bayes covers it

How deep it goes
Works through probability-weighted launch payoffs and sequential decisions, then identifies search cost and approximation strategies.
Examples
Uses a website launch with competing gains and losses, plus a restaurant choice between a familiar dish and a new one.
What it assumes
Builds utility and expected utility from a concrete choice while relying on posterior predictions to supply beliefs about outcomes.
What you can do after
Practice/perform action comparisons by weighting possible utilities by their probabilities and selecting the highest expected utility.
How it connects
Connects utility functions and posterior beliefs to sequential decision-making, the exploration–exploitation tradeoff, and Monte Carlo approximation.

Also explained in

Compare how 2 books explain decision theory →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Bayes. The section links point to the parts of the book it draws on.