Ranked List Truncation for Large Language Model-based Re-Ranking

arXiv:2404.18185v1 Announce Type: cross
Abstract: We study ranked list truncation (RLT) from a novel "retrieve-then-re-rank" perspective, where we optimize re-ranking by truncating the retrieved list (i.e., trim re-ranking candidates). RLT is crucial for re-ranking as it can improve re-ranking efficiency by sending variable-length candidate lists to a re-ranker on a per-query basis. It also has the potential to improve re-ranking effectiveness. Despite its importance, there is limited research into applying RLT methods to this new perspective. To address this research …

abstract arxiv cs.ai cs.cl cs.ir cs.lg efficiency language language model large language large language model list lists novel per perspective query ranking study type