An Information Minimization Based Contrastive Learning Model for Unsupervised Sentence Embeddings Learning. (arXiv:2209.10951v1 [cs.CL])

Unsupervised sentence embeddings learning has been recently dominated by
contrastive learning methods (e.g., SimCSE), which keep positive pairs similar
and push negative pairs apart. The contrast operation aims to keep as much
information as possible by maximizing the mutual information between positive
instances, which leads to redundant information in sentence embedding. To
address this problem, we present an information minimization based contrastive
learning (InforMin-CL) model to retain the useful information and discard the
redundant information by maximizing the mutual information …

arxiv information unsupervised