Hypothesis testing evaluates whether observed data provide convincing evidence for a claim. A classical test starts with a null hypothesis, models data under that assumption, and measures how unusual the observed result would be. A Bayesian framing instead compares competing models and can connect evidence to decisions. section 11.0 section 11.3
A traditional hypothesis test follows these steps:
- Specify a null hypothesis that defines the baseline model. section 11.3
- Choose a test statistic that summarizes the relevant data. section 11.3
- Calculate how often the null model produces a result at least as extreme as the observation; this probability is the p-value. section 11.2 section 11.3
- Reject the null when the p-value falls below a chosen significance level, often 0.05. section 11.3
Testing a website redesign
Suppose 63 of 500 visitors using a new design convert, an observed rate of 12.6%. Set the baseline conversion rate to 10% and test whether the redesign performs better. Under the null hypothesis, conversions follow a binomial distribution with probability 0.1. The test asks whether 63 or more conversions would be unusually high under that model. section 11.2
Use the compact equation to inspect the null hypothesis for the conversion-rate example. section 11.2
N = 500
n_conversions = 63
baseline = 0.1
test = binomtest(
n_conversions, #A
N, #B
baseline, #C
alternative="greater" #D
)
p_value = test.pvalueUse the Python listing to translate the binomial test into a p-value calculation. section 11.2







