What Hypothesis Test Should I Use for My A/B Experiment?

A quick-reference guide for choosing the correct test

The choice of hypothesis test after running an A/B experiment can be overwhelming for any data practitioner. Both scipy and statsmodels, two popular Python libraries, have implemented hundreds of hypothesis tests for all types of use cases.

This article is your quick reference guide to the most common types of hypothesis tests you’ll need to analyse your A/B(/C) experiments. I use these tests regularly …

a-b-testing causal inference data science experiment experimentation hypothesis hypothesis-testing test