Can Contrastive Learning Refine Embeddings

arXiv:2404.08701v1 Announce Type: new
Abstract: Recent advancements in contrastive learning have revolutionized self-supervised representation learning and achieved state-of-the-art performance on benchmark tasks. While most existing methods focus on applying contrastive learning to input data modalities such as images, natural language sentences, or networks, they overlook the potential of utilizing outputs from previously trained encoders. In this paper, we introduce SIMSKIP, a novel contrastive learning framework that specifically refines input embeddings for downstream tasks. Unlike traditional unsupervised learning approaches, SIMSKIP takes …

abstract art arxiv benchmark cs.lg data embeddings focus images language natural natural language networks performance refine representation representation learning state tasks type