Towards the Generalization of Contrastive Self-Supervised Learning. (arXiv:2111.00743v3 [cs.LG] UPDATED)

Recently, self-supervised learning has attracted great attention, since it
only requires unlabeled data for training. Contrastive learning is one popular
method for self-supervised learning and has achieved promising empirical
performance. However, the theoretical understanding of its generalization
ability is still limited. To this end, we define a kind of
$(\sigma,\delta)$-measure to mathematically quantify the data augmentation, and
then provide an upper bound of the downstream classification error based on the
measure. We show that the generalization ability of contrastive self-supervised …

arxiv learning self-supervised learning supervised learning