Evaluating the Robustness of Machine Reading Comprehension Models to Low Resource Entity Renaming

arXiv:2304.03145v2 Announce Type: replace
Abstract: Question answering (QA) models have shown compelling results in the task of Machine Reading Comprehension (MRC). Recently these systems have proved to perform better than humans on held-out test sets of datasets e.g. SQuAD, but their robustness is not guaranteed. The QA model's brittleness is exposed when evaluated on adversarial generated examples by a performance drop. In this study, we explore the robustness of MRC models to entity renaming, with entities from low-resource regions such …

abstract arxiv cs.cl datasets humans low machine question question answering reading results robustness systems test type