Distillation Enhanced Generative Retrieval

arXiv:2402.10769v1 Announce Type: new
Abstract: Generative retrieval is a promising new paradigm in text retrieval that generates identifier strings of relevant passages as the retrieval target. This paradigm leverages powerful generative language models, distinct from traditional sparse or dense retrieval methods. In this work, we identify a viable direction to further enhance generative retrieval via distillation and propose a feasible framework, named DGR. DGR utilizes sophisticated ranking models, such as the cross-encoder, in a teacher role to supply a passage …

abstract arxiv cs.ai cs.cl cs.ir distillation generative generative retrieval identify language language models new paradigm paradigm retrieval strings text type via work