A mathematical assumption is a condition that determines when a method’s results are meaningful and can be trusted. It does not produce the result; it sets the conditions under which the result can be evaluated. When those conditions fail, an output may still look convincing while resting on fragile foundations. section 1.2.4
Common conditions shape what conclusions a method can support:
- Independence of observations and stationarity of data help distinguish stable relationships from temporary patterns. section 1.2.4
- Functional-form and error assumptions matter for linear regression; neural networks relax some assumptions but require attention to data, optimization, and generalization. section 1.2.4
- Optimization conditions can determine whether a problem can be decomposed without losing the structure needed for a global solution. section 1.2.4
Choosing a statistical test
A test should match both the research question and the data’s assumptions. Violating those assumptions can make results unreliable; choosing a less appropriate test can also reduce power. The test should be selected before analyzing the data and kept fixed. section 4.1.3
Assumptions belong to a wider chain of design choices, from data and modeling frameworks to algorithms and evaluation. Final outputs alone may conceal these choices, so reliability requires examining the conditions and decisions that shaped them. section 1.2.3 section 1.2
Table 4.1 Statistical test selection table. Choosing the right statistical test depends on both the type of data and the research question. The table summarizes many of the most commonly used statistical tests but is not intended to be an exhaustive list of available procedures.
|
Research
Question
|
Common
Test(s)
|
Key
Assumptions
|
|
Compare
a sample mean to a known reference value (continuous outcome)
|
z-test
(one-sample)
|
Known
population variance or large sample size (normal approximation), independent
observations
|
|
Compare
means between 2 groups (continuous outcome)
|
t-test
(independent or paired, depending on design)
|
Normality,
equal variances, independent samples
|
|
Compare
means across 3+ groups (continuous outcome)
|
ANOVA
(or Kruskal-Wallis if non-normal)
|
Normality,
homogeneity of variance
|
|
Compare
proportions between 2 groups (categorical outcome)
|
Chi-square
test (or Fisher’s exact test for small samples)
|
Expected
cell counts reasonably large
|
|
Compare
proportions across 3+ groups (categorical outcome)
|
Chi-square
test of independence
|
Expected
cell counts reasonably large
|
|
Relationship
between 2 continuous variables
|
Correlation
(Pearson or Spearman) / simple linear regression
|
Linearity,
normality (for Pearson)
|
|
Relationship
between categorical and continuous variables
|
t-test,
ANOVA, or regression (depending on groups and design)
|
Normality,
equal variances (for parametric tests)
|
|
Goodness-of-fit
for categorical data
|
Chi-square
goodness-of-fit test
|
Independent
observations
|
Compare common statistical tests by research question, data type, and key assumptions. section 4.1.3


