chapter five
5 The Statistics You (Probably) Didn’t Learn: Non-Parametric Tests, Chi-Square Tests, and Responsible Interpretation
This chapter covers
- The history and original purpose of common statistical tests
- Evaluating and using non-parametric alternatives to common parametric tests
- Leveraging the chi-square test for categorical comparisons
- Mitigating the likelihood of false positive and false negative results
- Using statistics responsibly to ensure the accuracy of your findings
“The number of ways you can misunderstand statistics is infinite. The number of ways you can understand it is finite.”
– Dr. Lawrence Tatum
You are an analyst on a product analytics team at a software company. The product team is evaluating whether one of the new page versions on the app leads to customers completing a workflow faster. You were asked to conduct a between-subjects ANOVA to assess the results of an A/B/C test. In your diligence as an analyst, you start by exploring the distributions of data to check the assumption of normality. The distributions look like this:
Figure 5.1 The distribution of each group is bimodal.
You try transforming each distribution with a few of the recommended approaches discussed in chapter 4 but cannot change the shape of the bimodal distributions.
Figure 5.2 Square and square root transformations of the original bimodal distributions.
