liveBook · Manning

Statistical significance

As explained in Statistics Slam Dunk

Statistical significance is an inferential assessment that an observed result is unlikely under a specified null hypothesis at a chosen significance threshold.

Statistical significance asks whether an observed difference is unlikely to be due to chance under a null hypothesis. It depends on the difference between group means, variation within groups, and the number of records: a small difference can be significant in a long dataset, while a large difference may not be significant in a short one. A p-value at or below 0.05 is used as the cutoff for rejecting a null hypothesis of equal means. section 7.4.1 section 12.7.2

For comparisons of exactly two groups with continuous data, a t-test evaluates whether the means are essentially equal or different. Start with a clear null hypothesis, calculate the test, compare its p-value with the chosen threshold, and then either reject or fail to reject the null hypothesis. The result turns a visible difference into a more disciplined conclusion about whether chance is a plausible explanation. section 7.4.1 section 1.3

A practical significance-testing workflow is:

  • State the null hypothesis. For home and road teams, the starting claim is that their average fouls or free-throw attempts do not differ. section 7.4.1
  • Inspect the observed totals and average differences before testing, rather than treating the p-value as the whole story. section 7.4.1
  • Run a t-test when comparing two continuous groups, then check whether the p-value is at or below 0.05. section 7.4.1
  • Interpret the decision in context: reject the equal-means null when the result clears the threshold; otherwise fail to reject it. section 12.7.2

Home versus road foul calls

During the 2018–19 regular season, road teams averaged about 0.38 more personal fouls per game than home teams. A Welch two-sample t-test returned p = 0.02727, below 0.05, so the equal-means null hypothesis was rejected. The same analysis found home teams attempted about 0.65 more free throws per game, with p = 0.03072. Paired boxplots display the distributions and their mean markers alongside these testable differences. section 7.4.1 section 7.4.1

Salary inequality and championships

A second two-group comparison contrasts Gini coefficients for championship-winning teams and all other teams from 1991–2018. Champions averaged 0.53 versus 0.48 for other teams; the t-test produced p = 0.006245. Because this is below the 5% threshold, the difference in salary distributions is treated as statistically significant rather than merely chance variation. section 12.7.2

options(scipen = 999)

t.test(fouls1819reg$pfR, fouls1819reg$pfH)
## 
##  Welch Two Sample t-test
## 
## data:  fouls1819reg$pfR and fouls1819reg$pfH
## t = 2.2089, df = 2457.8, p-value = 0.02727
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  0.04290002 0.72132763
## sample estimates:
## mean of x mean of y 
##  21.09512  20.71301

This R output shows the full Welch two-sample t-test for road and home personal fouls, including the p-value, confidence interval, and group means used to interpret significance. section 7.4.1

How Statistics Slam Dunk covers it

How deep it goes
Defines significance through p-values, thresholds, means, variances, and record counts, then applies Welch two-sample t-tests without deriving their mathematics.
Examples
Uses home-versus-road fouls and free throws, plus championship-team salary Gini coefficients and standardized scoring and defense boxplots.
What it assumes
Introduces null hypotheses and p-value decisions in practical comparisons while assuming data prepared as two continuous groups.
What you can do after
Apply it via R’s t.test() to decide whether two-group mean differences meet a 5% significance threshold.
How it connects
Connects null hypotheses, p-values, t-tests, confidence intervals, effect size testing, Gini coefficients, boxplots, and standardized variables.

Also explained in

Compare how 7 books explain statistical significance →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Statistics Slam Dunk. The section links point to the parts of the book it draws on.