Mitigating Biases with Diverse Ensembles and Diffusion Models

arXiv:2311.16176v2 Announce Type: replace
Abstract: Spurious correlations in the data, where multiple cues are predictive of the target labels, often lead to a phenomenon known as shortcut bias, where a model relies on erroneous, easy-to-learn cues while ignoring reliable ones. In this work, we propose an ensemble diversification framework exploiting Diffusion Probabilistic Models (DPMs) for shortcut bias mitigation. We show that at particular training intervals, DPMs can generate images with novel feature combinations, even when trained on samples displaying correlated …

abstract arxiv bias biases correlations cs.ai cs.cv cs.lg data diffusion diffusion models diverse diversification easy ensemble framework labels learn multiple predictive shortcut type work