Reasoning over Description Logic-based Contexts with Transformers

arXiv:2311.08941v2 Announce Type: replace
Abstract: One way that the current state of the art measures the reasoning ability of transformer-based models is by evaluating accuracy in downstream tasks like logical question answering or proof generation over synthetic contexts expressed in natural language. However, most of the contexts used are in practice very simple; in most cases, they are generated from short first-order logic sentences with only a few logical operators and quantifiers. In this work, we seek to answer the …

abstract accuracy art arxiv cs.ai cs.cl current language logic natural natural language practice question question answering reasoning simple state state of the art synthetic tasks transformer transformers type