Assessing generalization capability of text ranking models in Polish

arXiv:2402.14318v1 Announce Type: new
Abstract: Retrieval-augmented generation (RAG) is becoming an increasingly popular technique for integrating internal knowledge bases with large language models. In a typical RAG pipeline, three models are used, responsible for the retrieval, reranking, and generation stages. In this article, we focus on the reranking problem for the Polish language, examining the performance of rerankers and comparing their results with available retrieval models. We conduct a comprehensive evaluation of existing models and those trained by us, utilizing …

abstract article arxiv capability cs.cl focus knowledge language language models large language large language models pipeline popular rag ranking responsible retrieval retrieval-augmented text text ranking type