ChatGPT Rates Natural Language Explanation Quality Like Humans: But on Which Scales?

arXiv:2403.17368v1 Announce Type: new
Abstract: As AI becomes more integral in our lives, the need for transparency and responsibility grows. While natural language explanations (NLEs) are vital for clarifying the reasoning behind AI decisions, evaluating them through human judgments is complex and resource-intensive due to subjectivity and the need for fine-grained ratings. This study explores the alignment between ChatGPT and human assessments across multiple scales (i.e., binary, ternary, and 7-Likert scale). We sample 300 data instances from three NLE datasets …

abstract ai decisions arxiv chatgpt cs.ai cs.cl decisions human humans integral language natural natural language quality reasoning responsibility them through transparency type vital