Don’t Use a T-Test for A/B Testing

How to use multiple linear regression to determine ATE and statistical significance

Have you ever wanted to speed up an A/B test? Well, here’s arguably the highest ROI solution for variance reduction in an A/B testing setting.

Frequentist experimentation is commonly leveraged for experimentation. However, as compared to Bayesian or sequential regimes, frequentist A/B tests often require large sample sizes, which slows down company iteration. Below we will discuss a practical solution that allows …

ab-test a/b testing b testing causal inference data science experiment machine learning test testing t-test