Evaluating Generative Ad Hoc Information Retrieval

arXiv:2311.04694v2 Announce Type: replace-cross
Abstract: Recent advances in large language models have enabled the development of viable generative retrieval systems. Instead of a traditional document ranking, many generative retrieval systems directly return a grounded generated text as an answer to an information need expressed as a query or question. Quantifying the utility of the textual responses is essential for appropriately evaluating such generative ad hoc retrieval. Yet, the established evaluation methodology for ranking-based retrieval is not suited for reliable, repeatable, …

abstract advances arxiv cs.cl cs.ir development document generated generative generative retrieval information language language models large language large language models query question ranking retrieval systems text type utility