Sample size planning for conditional counterfactual mean estimation with a K-armed randomized experiment

arXiv:2403.04039v1 Announce Type: new
Abstract: We cover how to determine a sufficiently large sample size for a $K$-armed randomized experiment in order to estimate conditional counterfactual expectations in data-driven subgroups. The sub-groups can be output by any feature space partitioning algorithm, including as defined by binning users having similar predictive scores or as defined by a learned policy tree. After carefully specifying the inference target, a minimum confidence level, and a maximum margin of error, the key is to turn …

abstract algorithm arxiv counterfactual cs.lg data data-driven experiment feature mean partitioning planning sample space stat.me stat.ml subgroups type