MoleculeQA: A Dataset to Evaluate Factual Accuracy in Molecular Comprehension

arXiv:2403.08192v1 Announce Type: new
Abstract: Large language models are playing an increasingly significant role in molecular research, yet existing models often generate erroneous information, posing challenges to accurate molecular comprehension. Traditional evaluation metrics for generated content fail to assess a model's accuracy in molecular understanding. To rectify the absence of factual evaluation, we present MoleculeQA, a novel question answering (QA) dataset which possesses 62K QA pairs over 23K molecules. Each QA pair, composed of a manual question, a positive option …

abstract accuracy arxiv challenges cs.cl dataset evaluation evaluation metrics generate generated information language language models large language large language models metrics playing q-bio.bm research role type understanding