Variance reduction on steroids… introducing MLRATE

Upgrade Variance Reduction Beyond CUPED: Introducing MLRATE

MLRATE uses machine learning to reach statistical power more quickly when experimenting.

Say you are running an experiment, and there is a true difference in the means between your control and variant population. Will you detect it? That depends on three things:

1. The size of the effect
2. The number of your samples you collect
3. The variance of your measurement

The size of your effect isn’t often something you can alter. Increasing sample …

data science decision-science machine learning notes-from-industry variance