Counterfactual contrastive learning: robust representations via causal image synthesis

arXiv:2403.09605v1 Announce Type: new
Abstract: Contrastive pretraining is well-known to improve downstream task performance and model generalisation, especially in limited label settings. However, it is sensitive to the choice of augmentation pipeline. Positive pairs should preserve semantic information while destroying domain-specific information. Standard augmentation pipelines emulate domain-specific changes with pre-defined photometric transformations, but what if we could simulate realistic domain changes instead? In this work, we show how to utilise recent progress in counterfactual image generation to this effect. We …

abstract arxiv augmentation causal counterfactual cs.ai cs.cv domain however image information performance pipeline pipelines positive pretraining robust semantic standard synthesis type via