AEON: A Method for Automatic Evaluation of NLP Test Cases. (arXiv:2205.06439v1 [cs.SE])

Due to the labor-intensive nature of manual test oracle construction, various
automated testing techniques have been proposed to enhance the reliability of
Natural Language Processing (NLP) software. In theory, these techniques mutate
an existing test case (e.g., a sentence with its label) and assume the
generated one preserves an equivalent or similar semantic meaning and thus, the
same label. However, in practice, many of the generated test cases fail to
preserve similar semantic meaning and are unnatural (e.g., grammar errors), …

arxiv cases evaluation nlp test