Can citations tell us about a paper's reproducibility? A case study of machine learning papers

arXiv:2405.03977v1 Announce Type: cross
Abstract: The iterative character of work in machine learning (ML) and artificial intelligence (AI) and reliance on comparisons against benchmark datasets emphasize the importance of reproducibility in that literature. Yet, resource constraints and inadequate documentation can make running replications particularly challenging. Our work explores the potential of using downstream citation contexts as a signal of reproducibility. We introduce a sentiment analysis framework applied to citation contexts from papers involved in Machine Learning Reproducibility Challenges in order …

arxiv case case study citations cs.ai cs.dl cs.lg machine machine learning paper papers reproducibility study type