Multi-Task Contrastive Learning for 8192-Token Bilingual Text Embeddings

arXiv:2402.17016v1 Announce Type: new
Abstract: We introduce a novel suite of state-of-the-art bilingual text embedding models that are designed to support English and another target language. These models are capable of processing lengthy text inputs with up to 8192 tokens, making them highly versatile for a range of natural language processing tasks such as text retrieval, clustering, and semantic textual similarity (STS) calculations.
By focusing on bilingual models and introducing a unique multi-task learning objective, we have significantly improved the …

abstract art arxiv bilingual cs.ai cs.cl cs.ir embedding embedding models embeddings english inputs language language processing making natural natural language natural language processing novel processing state support tasks text text embedding them token tokens type