Frequentist hypothesis testing makes inferences about a broader population from samples. It typically frames hypotheses as educated guesses about group differences or their direction, without specifying a numerical effect size. section 3.1.3
Quantifying the expected difference belongs more closely to Bayesian hypothesis testing, which treats probability as a degree of belief based on available evidence. Estimating that difference can draw on research, organizational data, or stakeholder knowledge. section 3.1.3
Treatment and control distributions
A Python simulation generates overlapping normal distributions for control and treatment groups, shifts treatment two standard deviations to the right, and displays both with overlaid histograms. section 3.1.3
import numpy as np #1 import matplotlib.pyplot as plt c = np.random.normal(0, 1, size=500) #2 t = np.random.normal(2, 1, size=500) plt.hist(c, alpha=0.5, bins=25, color="black") #3 plt.hist(t, alpha=0.5, bins=25, color="lightgray") plt.legend(["Control", "Treatment"])
Run the Python listing to simulate and plot hypothetical treatment and control distributions. section 3.1.3
