Leveraging Diffusion Disentangled Representations to Mitigate Shortcuts in Underspecified Visual Tasks. (arXiv:2310.02230v4 [cs.CV] UPDATED)

Spurious correlations in the data, where multiple cues are predictive of the
target labels, often lead to shortcut learning phenomena, where a model may
rely on erroneous, easy-to-learn, cues while ignoring reliable ones. In this
work, we propose an ensemble diversification framework exploiting the
generation of synthetic counterfactuals using Diffusion Probabilistic Models
(DPMs). We discover that DPMs have the inherent capability to represent
multiple visual cues independently, even when they are largely correlated in
the training data. We leverage this …

arxiv correlations data diffusion diversification easy ensemble framework labels learn multiple predictive shortcut tasks visual work