GISTEmbed: Guided In-sample Selection of Training Negatives for Text Embedding Fine-tuning

arXiv:2402.16829v1 Announce Type: cross
Abstract: Embedding models are integral to AI applications like semantic search, personalized recommendations, and retrieval augmented generation for LLMs, necessitating high-quality training data. However, the limited scalability of manual data curation prompts the need for automated methods to ensure data integrity. Traditional unsupervised triplet mining automates training data generation, crucial for embedding model training, yet inadvertently injects biases and noise, thereby degrading model performance. Addressing this, we introduce GISTEmbed, a novel strategy that enhances in-batch negative …

abstract ai applications applications arxiv automated cs.cl cs.lg curation data data curation data integrity embedding embedding models fine-tuning integral integrity llms mining personalized personalized recommendations prompts quality recommendations retrieval retrieval augmented generation sample scalability search semantic text text embedding training training data type unsupervised