Pcc-tuning: Breaking the Contrastive Learning Ceiling in Semantic Textual Similarity

arXiv:2406.09790v1 Announce Type: new
Abstract: Semantic Textual Similarity (STS) constitutes a critical research direction in computational linguistics and serves as a key indicator of the encoding capabilities of embedding models. Driven by advances in pre-trained language models and contrastive learning techniques, leading sentence representation methods can already achieved average Spearman's correlation scores of approximately 86 across seven STS benchmarks in SentEval. However, further improvements have become increasingly marginal, with no existing method attaining an average score higher than 87 on …

abstract advances arxiv breaking capabilities computational cs.cl embedding embedding models encoding key language language models linguistics pcc representation research semantic textual tuning type