liveBook · Manning

Statistical significance

As explained in Timeless Algorithms, The Foundational Ideas

Statistical significance is an assessment of whether an observed result is sufficiently unlikely under a specified null hypothesis.

Statistical significance helps interpret a hypothesis-test decision, but it does not by itself establish that a finding matters in practice. Conclusions should also consider effect size, domain knowledge, costs and benefits, and the investigation’s objectives. section 4.1.7

A small clinical trial

Researchers test whether a drug raises recovery above a 50% baseline, using α = 0.05. When the evidence falls just short of that threshold, they fail to reject the null hypothesis—but do not prove it true. The result calls for judgment and may motivate further study. section 4.2.3

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Applies a predetermined threshold to an ambiguous trial result and distinguishes non-rejection from proof of the null.
Examples
Uses a small trial testing whether a drug raises recovery above a historical 50% baseline.
What you can do after
Evaluate conclusions by relating test decisions to assumptions, data quality, study scope, and practical stakes.
How it connects
Links hypothesis testing to null hypotheses, significance levels, Type I and Type II errors, and practical significance.

Also explained in

Compare how 9 books explain statistical significance →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.