How much reliable is ChatGPT's prediction on Information Extraction under Input Perturbations?

arXiv:2404.05088v1 Announce Type: new
Abstract: In this paper, we assess the robustness (reliability) of ChatGPT under input perturbations for one of the most fundamental tasks of Information Extraction (IE) i.e. Named Entity Recognition (NER). Despite the hype, the majority of the researchers have vouched for its language understanding and generation capabilities; a little attention has been paid to understand its robustness: How the input-perturbations affect 1) the predictions, 2) the confidence of predictions and 3) the quality of rationale behind …

abstract arxiv chatgpt cs.cl extraction hype information information extraction language language understanding ner paper prediction recognition reliability researchers robustness tasks type understanding