ShortcutLens: A Visual Analytics Approach for Exploring Shortcuts in Natural Language Understanding Dataset. (arXiv:2208.08010v1 [cs.HC])

Benchmark datasets play an important role in evaluating Natural Language
Understanding (NLU) models. However, shortcuts -- unwanted biases in the
benchmark datasets -- can damage the effectiveness of benchmark datasets in
revealing models' real capabilities. Since shortcuts vary in coverage,
productivity, and semantic meaning, it is challenging for NLU experts to
systematically understand and avoid them when creating benchmark datasets. In
this paper, we develop a visual analytics system, ShortcutLens, to help NLU
experts explore shortcuts in NLU benchmark datasets. …

analytics arxiv dataset language natural natural language understanding visual analytics