Not another Negation Benchmark: The NaN-NLI Test Suite for Sub-clausal Negation. (arXiv:2210.03256v2 [cs.CL] UPDATED)

Negation is poorly captured by current language models, although the extent
of this problem is not widely understood. We introduce a natural language
inference (NLI) test suite to enable probing the capabilities of NLP methods,
with the aim of understanding sub-clausal negation. The test suite contains
premise--hypothesis pairs where the premise contains sub-clausal negation and
the hypothesis is constructed by making minimal modifications to the premise in
order to reflect different possible interpretations. Aside from adopting
standard NLI labels, our …

arxiv benchmark nan test