all AI news
Are All the Datasets in Benchmark Necessary? A Pilot Study of Dataset Evaluation for Text Classification. (arXiv:2205.02129v1 [cs.CL])
cs.CL updates on arXiv.org arxiv.org
In this paper, we ask the research question of whether all the datasets in
the benchmark are necessary. We approach this by first characterizing the
distinguishability of datasets when comparing different systems. Experiments on
9 datasets and 36 systems show that several existing benchmark datasets
contribute little to discriminating top-scoring systems, while those less used
datasets exhibit impressive discriminative power. We further, taking the text
classification task as a case study, investigate the possibility of predicting
dataset discrimination based on …
arxiv benchmark classification dataset datasets evaluation pilot study text text classification