FroSSL: Frobenius Norm Minimization for Efficient Multiview Self-Supervised Learning

arXiv:2310.02903v3 Announce Type: replace
Abstract: Self-supervised learning (SSL) is a popular paradigm for representation learning. Recent multiview methods can be classified as sample-contrastive, dimension-contrastive, or asymmetric network-based, with each family having its own approach to avoiding informational collapse. While these families converge to solutions of similar quality, it can be empirically shown that some methods are epoch-inefficient and require longer training to reach a target performance. Two main approaches to improving efficiency are covariance eigenvalue regularization and using more views. …

abstract arxiv converge cs.lg families family network norm paradigm popular quality representation representation learning sample self-supervised learning solutions ssl supervised learning type