Evaluating Unsupervised Dimensionality Reduction Methods for Pretrained Sentence Embeddings

arXiv:2403.14001v1 Announce Type: new
Abstract: Sentence embeddings produced by Pretrained Language Models (PLMs) have received wide attention from the NLP community due to their superior performance when representing texts in numerous downstream applications. However, the high dimensionality of the sentence embeddings produced by PLMs is problematic when representing large numbers of sentences in memory- or compute-constrained devices. As a solution, we evaluate unsupervised dimensionality reduction methods to reduce the dimensionality of sentence embeddings produced by PLMs. Our experimental results show …

abstract applications arxiv attention community cs.cl dimensionality embeddings however language language models nlp numbers performance type unsupervised