Hypothesis testing
Hypothesis testing is a formal statistical procedure that uses observed data to evaluate evidence about a specified hypothesis, often a null hypothesis.
Grokking Statistics
A statistical procedure for evaluating evidence against a specified null hypothesis about a population or data-generating process.
- How it is used
- Tests sample-based claims about population means, proportions, and variances for practical decisions.
- How deep it goes
- Works through hypotheses, tails, α, p-values, critical regions, and small-sample t-tests without formal derivations.
- Examples
- Uses soda-can volume, tea identification, and a college exam-score sample to ground testing decisions.
- What you can do after
- Apply it via Python and SciPy to select tails, calculate p-values, and decide whether to reject H0.
Causal AI
A hypothesis test is a statistical procedure for evaluating whether observed data provide sufficient evidence against a specified null hypothesis.
- How it is used
- Tests observed group differences, DAG-implied conditional independences, causal effect estimates, and assumptions behind those estimates.
- How deep it goes
- Works through null hypotheses, statistics, significance thresholds, and Python tests, while treating rigorous independence-as-alternative testing as mathematically nontrivial.
- Examples
- Uses transportation-survey chi-squared tests and an online-game comparison of purchases between high- and low-engagement players.
- What you can do after
- Apply it via pgmpy to challenge DAG independences and via DoWhy to test whether simulated assumption violations refute an estimate.
Statistics Every Programmer Needs
A hypothesis test is a statistical procedure for evaluating evidence against a specified null hypothesis.
- How it is used
- Uses statistical tests to assess evidence against a null hypothesis, including checks of variable normality before simple regression.
- How deep it goes
- Defines p-values and significance decisions, and discusses the conventional threshold and its false-positive and false-negative trade-off.
- Examples
- Runs Shapiro–Wilk tests on stage1 and stage2 variables and reports each test statistic and p-value.
- What you can do after
- Evaluate evidence by comparing a test's p-value with a predetermined threshold and deciding whether to reject or fail to reject the null.
Read how Statistics Every Programmer Needs explains hypothesis testing →
Related concepts
- t-testincludes · A t-test is a statistical hypothesis test that uses a t-distributed statistic to assess a mean or difference in means under specified assumptions.
- chi-square testincludes · A chi-square test is a statistical hypothesis test that uses a chi-square-distributed statistic to evaluate hypotheses about categorical data, goodness of fit, or variance.
- p-valueuses · A p-value is the probability, assuming a specified null hypothesis, of obtaining a test statistic or result at least as extreme as the observed one.
- null hypothesisuses · A null hypothesis is a designated baseline statistical claim, typically asserting that an effect, difference, or association is absent.
- correlation testingincludes · Correlation testing is a statistical procedure for evaluating the association between variables, often against a specified null value.
- f-testincludes · An F-test is a statistical hypothesis test using an F-distributed ratio of variance estimates to compare variances or assess group differences and model effects.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.