Self-supervised Representation Learning From Random Data Projectors

arXiv:2310.07756v2 Announce Type: replace
Abstract: Self-supervised representation learning~(SSRL) has advanced considerably by exploiting the transformation invariance assumption under artificially designed data augmentations. While augmentation-based SSRL algorithms push the boundaries of performance in computer vision and natural language processing, they are often not directly applicable to other data modalities, and can conflict with application-specific data augmentation constraints. This paper presents an SSRL approach that can be applied to any data modality and network architecture because it does not rely on augmentations …

abstract advanced algorithms and natural language processing application arxiv augmentation computer computer vision conflict cs.lg data language language processing natural natural language natural language processing performance processing random representation representation learning transformation type vision