A post hoc test is a set of follow-up comparisons used after a test of three or more groups finds a significant overall difference. The overall result says that groups differ, but not which pairs differ. Post hoc comparisons answer that narrower question. section 5.4 section 5.2.1
Testing every pair creates more opportunities for a false positive. A post hoc procedure adjusts for the number of comparisons so that an isolated small p-value is not treated as stronger evidence than it is. The adjustment matters when reporting which groups differ to stakeholders. section 5.2.1 section 5.4
Match the follow-up to the overall test and the data:
- For continuous or ordinal data compared with a Kruskal–Wallis test, use a procedure such as Dunn’s test. A Bonferroni adjustment is one way to account for the multiple pairs. section 5.2.1
- For a significant chi-square result on categorical data, pairwise 2 × 2 comparisons can help locate differences between groups. Check expected cell frequencies; the overall chi-square result does not identify the responsible pairs. section 5.2.3
Three-group comparison
Suppose three groups of scores are compared with Kruskal–Wallis. After a significant overall result, `scikit_posthocs.posthoc_dunn(data, p_adjust="bonferroni")` produces adjusted p-values for the group pairs. Use those values to identify supported differences, then inspect the groups’ scores to describe which is higher; the Kruskal–Wallis H-statistic does not give the direction. section 5.2.1
A pairwise result is not automatically a multiple-comparison-adjusted result. In particular, a loop that reruns chi-square tests for each pair shows where differences may lie, but its printed p-values should not be described as familywise-adjusted without a separate adjustment. Report non-significant findings too. section 5.2.3 section 5.4
