Simpler becomes Harder: Do LLMs Exhibit a Coherent Behavior on Simplified Corpora?

arXiv:2404.06838v1 Announce Type: new
Abstract: Text simplification seeks to improve readability while retaining the original content and meaning. Our study investigates whether pre-trained classifiers also maintain such coherence by comparing their predictions on both original and simplified inputs. We conduct experiments using 11 pre-trained models, including BERT and OpenAI's GPT 3.5, across six datasets spanning three languages. Additionally, we conduct a detailed analysis of the correlation between prediction change rates and simplification types/strengths. Our findings reveal alarming inconsistencies across all …

abstract arxiv behavior bert classifiers cs.cl inputs llms meaning predictions pre-trained models readability simplified study text type