On Linear Separation Capacity of Self-Supervised Representation Learning

arXiv:2310.19041v2 Announce Type: replace-cross
Abstract: Recent advances in self-supervised learning have highlighted the efficacy of data augmentation in learning data representation from unlabeled data. Training a linear model atop these enhanced representations can yield an adept classifier. Despite the remarkable empirical performance, the underlying mechanisms that enable data augmentation to unravel nonlinear data structures into linearly separable representations remain elusive. This paper seeks to bridge this gap by investigating under what conditions learned representations can linearly separate manifolds when data …

abstract adept advances arxiv augmentation capacity classifier cs.lg data linear linear model math.st performance representation representation learning self-supervised learning stat.ml stat.th supervised learning training type