PIRB: A Comprehensive Benchmark of Polish Dense and Hybrid Text Retrieval Methods

arXiv:2402.13350v1 Announce Type: new
Abstract: We present Polish Information Retrieval Benchmark (PIRB), a comprehensive evaluation framework encompassing 41 text information retrieval tasks for Polish. The benchmark incorporates existing datasets as well as 10 new, previously unpublished datasets covering diverse topics such as medicine, law, business, physics, and linguistics. We conduct an extensive evaluation of over 20 dense and sparse retrieval models, including the baseline models trained by us as well as other available Polish and multilingual methods. Finally, we introduce …

abstract arxiv benchmark business cs.cl datasets diverse evaluation framework hybrid information law linguistics medicine physics retrieval tasks text topics type