A Bias-Variance Decomposition for Ensembles over Multiple Synthetic Datasets

Recent studies have highlighted the benefits of generating multiple synthetic datasets for supervised learning, from increased accuracy to more effective model selection and uncertainty estimation. These benefits have clear empirical support, but the theoretical understanding of them is currently very light. We seek to increase the theoretical understanding by deriving bias-variance decompositions for several settings of using multiple synthetic datasets. Our theory predicts multiple synthetic datasets to be especially beneficial for high-variance downstream predictors, and yields a simple rule of …

accuracy benefits bias bias-variance clear cs.lg datasets light model selection multiple stat.ml studies supervised learning support synthetic them uncertainty understanding variance