SemEval-2024 Task 9: BRAINTEASER: A Novel Task Defying Common Sense

arXiv:2404.16068v1 Announce Type: cross
Abstract: While vertical thinking relies on logical and commonsense reasoning, lateral thinking requires systems to defy commonsense associations and overwrite them through unconventional thinking. Lateral thinking has been shown to be challenging for current models but has received little attention. A recent benchmark, BRAINTEASER, aims to evaluate current models' lateral thinking ability in a zero-shot setting. In this paper, we split the original benchmark to also support fine-tuning setting and present SemEval Task 9: BRAIN-TEASER(S), the …

abstract arxiv attention benchmark common sense commonsense cs.ai cs.cl cs.lg current novel reasoning sense systems them thinking through type while