What To Do When A/B Test Result Is Not Significant?

What to Do When the A/B Test Result Is Not Significant

4-step Framework to Land on Concrete Actions

Too Long; Don’t Read

• Step 1: Check Experiment Set-up & Data Correctness
• Step 2: Check if Sample Size and Statistical Power are Sufficient: if the collected samples satisfy the minimum required sample size, conduct nonparametric tests and assess if the experiment is underpowered. If the traffic is insufficient, keep waiting since formulating inference hypotheses based on …

a-b-testing data science product management significance statistics test