Word Embeddings Revisited: Do LLMs Offer Something New?

arXiv:2402.11094v1 Announce Type: new
Abstract: Learning meaningful word embeddings is key to training a robust language model. The recent rise of Large Language Models (LLMs) has provided us with many new word/sentence/document embedding models. Although LLMs have shown remarkable advancement in various NLP tasks, it is still unclear whether the performance improvement is merely because of scale or whether underlying embeddings they produce significantly differ from classical encoding models like Sentence-BERT (SBERT) or Universal Sentence Encoder (USE). This paper systematically …

abstract advancement arxiv cs.cl document embedding embedding models embeddings improvement key language language model language models large language large language models llms nlp performance robust something tasks training type word word embeddings