A Dataset for Physical and Abstract Plausibility and Sources of Human Disagreement

arXiv:2404.04035v1 Announce Type: new
Abstract: We present a novel dataset for physical and abstract plausibility of events in English. Based on naturally occurring sentences extracted from Wikipedia, we infiltrate degrees of abstractness, and automatically generate perturbed pseudo-implausible events. We annotate a filtered and balanced subset for plausibility using crowd-sourcing, and perform extensive cleansing to ensure annotation quality. In-depth quantitative analyses indicate that annotators favor plausibility over implausibility and disagree more on implausible events. Furthermore, our plausibility dataset is the first …

abstract arxiv cs.cl dataset english events generate human novel type wikipedia