liveBook · Manning

Hypothesis testing

As explained in Statistics Every Programmer Needs

A hypothesis test is a statistical procedure for evaluating evidence against a specified null hypothesis.

Hypothesis testing evaluates evidence against a specified null hypothesis. A p-value quantifies how likely the observed results are under that null; compare it with a predetermined significance threshold to decide whether to reject or fail to reject the null. section 4.2.1 section 1.1.4

For a normality check before simple regression:

  • Use a formal test rather than relying only on plots, which can mislead about whether data are normally distributed. section 4.2.1
  • Run Shapiro–Wilk tests on the variables and compare their p-values with a chosen significance threshold. section 4.2.1
  • Treat 5% as a common convention, not a universal rule; choose a clear threshold to guide decisions and balance false positives against false negatives. section 4.2.1
>>> from scipy import stats

>>> print(stats.shapiro(mds.stage1))
ShapiroResult(statistic=0.9446673217389622, pvalue=0.29328758849732417)

>>> print(stats.shapiro(mds.stage2))
ShapiroResult(statistic=0.9290259439587454, pvalue=0.1478907551379664)

Shapiro–Wilk results for the two variables report test statistics and p-values. section 4.2.1

How Statistics Every Programmer Needs covers it

How deep it goes
Defines p-values and significance decisions, and discusses the conventional threshold and its false-positive and false-negative trade-off.
Examples
Runs Shapiro–Wilk tests on stage1 and stage2 variables and reports each test statistic and p-value.
What you can do after
Evaluate evidence by comparing a test's p-value with a predetermined threshold and deciding whether to reject or fail to reject the null.
How it connects
Connects hypothesis tests with inferential statistics, p-values, null hypotheses, normality testing, and regression.

Also explained in

Compare how 4 books explain hypothesis testing →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Statistics Every Programmer Needs. The section links point to the parts of the book it draws on.