liveBook · Manning

Statistical significance

As explained in Grokking Statistics

A result property indicating that an observed pattern would be relatively unlikely under a specified null model.

Statistical significance describes an observed pattern that would be relatively unlikely under a specified null model. It can make a claim worth investigating, but it does not automatically show that one factor caused the result. section 1.4

A successful pricing system—or a departing competitor?

A director claims that a new pricing system boosted revenue by $70 million because an analysis found statistical significance. That result may be real, yet a competitor’s retreat from several markets could have delivered the extra customers instead. Statistical significance cannot replace checking alternative explanations. section 1.4

Read the number alongside sampling bias, process assumptions, ignored variables, aggressive outlier removal, logical fallacies, and the incentives behind the study. Statistical results can be shaped toward a desired finding or pressured into revision. section 1.4

How Grokking Statistics covers it

How deep it goes
Treats statistical significance as an interpretive claim to question, without supplying calculation or testing procedures.
Examples
Uses a pricing-system revenue claim and a cereal diabetes study to expose misleading interpretation and pressure.
What you can do after
Evaluate statistical claims by checking alternative explanations, ignored variables, outlier removal, and incentives.
How it connects
Connects statistical significance to sampling bias, process assumptions, outliers, logical fallacies, and confounding variables.

Also explained in

Compare how 7 books explain statistical significance →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Statistics. The section links point to the parts of the book it draws on.