liveBook · Manning

Decision theory

As explained in Timeless Algorithms, The Foundational Ideas

The formal study of how decision-makers choose among actions under uncertainty and in light of preferences or objectives.

In hypothesis testing, decision theory turns statistical evidence into action through rules established before the data is analyzed. The rule makes a decision systematic and reproducible, while acknowledging that uncertainty and the possibility of error remain. section 4.5 section 4.3.1

A Neyman–Pearson decision has two possible outcomes:

  • Reject the null hypothesis, risking a Type I error if it is true. section 4.1.6
  • Fail to reject the null hypothesis, risking a Type II error if a real effect goes undetected. Neither outcome proves its hypothesis true. section 4.1.6

A threshold does not erase nuance

With α = 0.05, p-values of 0.049 and 0.051 fall on opposite sides of the decision threshold despite providing nearly identical evidence against the null hypothesis. section 4.1.6

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Defines binary reject-or-fail-to-reject outcomes and explains Type I and II risks, without treating either outcome as certainty.
Examples
Compares p-values of 0.049 and 0.051 against a significance threshold of 0.05.
What you can do after
Evaluate an observed statistic or p-value against a predefined criterion without treating the result as proof.
How it connects
Links hypothesis testing to statistical inference, p-values, significance levels, statistical power, and Type I and Type II errors.

Also explained in

Compare how 2 books explain decision theory →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.