Beyond One-Size-Fits-All: Multi-Domain, Multi-Task Framework for Embedding Model Selection

arXiv:2404.00458v1 Announce Type: new
Abstract: This position paper proposes a systematic approach towards developing a framework to help select the most effective embedding models for natural language processing (NLP) tasks, addressing the challenge posed by the proliferation of both proprietary and open-source encoder models.

abstract arxiv beyond challenge cs.cl cs.ir domain embedding embedding models encoder framework language language processing model selection natural natural language natural language processing nlp paper processing proprietary tasks type