Can We Break Free from Strong Data Augmentations in Self-Supervised Learning?

arXiv:2404.09752v1 Announce Type: cross
Abstract: Self-supervised learning (SSL) has emerged as a promising solution for addressing the challenge of limited labeled data in deep neural networks (DNNs), offering scalability potential. However, the impact of design dependencies within the SSL framework remains insufficiently investigated. In this study, we comprehensively explore SSL behavior across a spectrum of augmentations, revealing their crucial role in shaping SSL model performance and learning mechanisms. Leveraging these insights, we propose a novel learning approach that integrates prior …

abstract arxiv challenge cs.ai cs.cv cs.lg data dependencies design explore framework free however impact networks neural networks scalability self-supervised learning solution ssl study supervised learning type