Planning Ahead in Generative Retrieval: Guiding Autoregressive Generation through Simultaneous Decoding

arXiv:2404.14600v1 Announce Type: cross
Abstract: This paper introduces PAG-a novel optimization and decoding approach that guides autoregressive generation of document identifiers in generative retrieval models through simultaneous decoding. To this aim, PAG constructs a set-based and sequential identifier for each document. Motivated by the bag-of-words assumption in information retrieval, the set-based identifier is built on lexical tokens. The sequential identifier, on the other hand, is obtained via quantizing relevance-based representations of documents. Extensive experiments on MSMARCO and TREC Deep Learning …

abstract aim arxiv autoregressive bag cs.cl cs.ir decoding document generative generative retrieval guides information novel optimization paper planning retrieval set through type words