Enhancing Embedding Performance through Large Language Model-based Text Enrichment and Rewriting

arXiv:2404.12283v1 Announce Type: new
Abstract: Embedding models are crucial for various natural language processing tasks but can be limited by factors such as limited vocabulary, lack of context, and grammatical errors. This paper proposes a novel approach to improve embedding performance by leveraging large language models (LLMs) to enrich and rewrite input text before the embedding process. By utilizing ChatGPT 3.5 to provide additional context, correct inaccuracies, and incorporate metadata, the proposed method aims to enhance the utility and accuracy …

abstract arxiv context cs.cl embedding embedding models errors language language model language models language processing large language large language model large language models llms natural natural language natural language processing novel paper performance processing tasks text through type