REASONS: A benchmark for REtrieval and Automated citationS Of scieNtific Sentences using Public and Proprietary LLMs

arXiv:2405.02228v1 Announce Type: new
Abstract: Automatic citation generation for sentences in a document or report is paramount for intelligence analysts, cybersecurity, news agencies, and education personnel. In this research, we investigate whether large language models (LLMs) are capable of generating references based on two forms of sentence queries: (a) Direct Queries, LLMs are asked to provide author names of the given research article, and (b) Indirect Queries, LLMs are asked to provide the title of a mentioned article when given …

abstract analysts arxiv automated benchmark citations cs.ai cs.cl cs.ir cybersecurity document education intelligence language language models large language large language models llms proprietary public report research retrieval scientific type