Wish Tackles Peeking with Always Valid p-values

Peek and conclude experiments adaptively without inflating the false-positive rate

Do you estimate the effect size and calculate the sample size before running your A/B test? Do you check the p-value only once when you conclude the A/B test? When the p-value is 5.1% (0.1% higher than the threshold 5%) and the pre-calculated sample size has been reached, instead of running the experiment longer, do you stop the experiment and conclude that …

a-b-testing data science ecommerce experimentation p-value values