Augmentations vs Algorithms: What Works in Self-Supervised Learning

arXiv:2403.05726v1 Announce Type: new
Abstract: We study the relative effects of data augmentations, pretraining algorithms, and model architectures in Self-Supervised Learning (SSL). While the recent literature in this space leaves the impression that the pretraining algorithm is of critical importance to performance, understanding its effect is complicated by the difficulty in making objective and direct comparisons between methods. We propose a new framework which unifies many seemingly disparate SSL methods into a single shared template. Using this framework, we identify …

abstract algorithm algorithms architectures arxiv cs.cv cs.lg data effects importance literature making performance pretraining self-supervised learning space ssl study supervised learning type understanding