liveBook · Manning

Experimentation

As explained in Timeless Algorithms, The Foundational Ideas

An experiment is an information-gathering procedure that can reveal something about uncertain states before a decision.

An experiment gathers information before a final act is chosen. Its evidence may update beliefs about uncertain states and change which act is preferred. Because experiments can impose costs, delays, or risks, their expected decision value must be weighed against those burdens. section 7.2 section 7.2.1

Information-gathering experiments

Diagnostic tests, market surveys, pilot programs, and prototype trials can provide evidence before a decision. section 7.2.1

Figure 7.3 Raiffa and Schlaifer’s decision-analytic cycle. Prior beliefs describe uncertainty about the possible states of the world. An experiment generates data that produces posterior beliefs, which inform the selection of an act. The resulting outcome is evaluated through utility and may provide the starting point for another decision.

The decision-analytic cycle links prior beliefs, experimental evidence, posterior beliefs, and act selection. section 7.2.1

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Defines experiments within acts, states, and experiments; explains belief updates and cost tradeoffs, but gives no experiment-design procedure.
Examples
Uses diagnostic tests, market surveys, pilot programs, and prototype trials to illustrate information-gathering experiments.
What you can do after
Evaluate whether to acquire information by weighing its expected decision value against costs, delays, or risks.
How it connects
Connects experiments to Bayesian updating, expected utility, decision analysis, and value-of-information analysis.

Also explained in

Compare how 20 books explain experimentation →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.