On Improving the Algorithm-, Model-, and Data- Efficiency of Self-Supervised Learning

arXiv:2404.19289v1 Announce Type: cross
Abstract: Self-supervised learning (SSL) has developed rapidly in recent years. However, most of the mainstream methods are computationally expensive and rely on two (or more) augmentations for each image to construct positive pairs. Moreover, they mainly focus on large models and large-scale datasets, which lack flexibility and feasibility in many practical applications. In this paper, we propose an efficient single-branch SSL method based on non-parametric instance discrimination, aiming to improve the algorithm, model, and data efficiency …

abstract algorithm arxiv construct cs.cv cs.lg data datasets efficiency focus however image improving large models positive scale self-supervised learning ssl supervised learning the algorithm type