Are self-explanations from Large Language Models faithful?

arXiv:2401.07927v3 Announce Type: replace-cross
Abstract: Instruction-tuned Large Language Models (LLMs) excel at many tasks and will even explain their reasoning, so-called self-explanations. However, convincing and wrong self-explanations can lead to unsupported confidence in LLMs, thus increasing risk. Therefore, it's important to measure if self-explanations truly reflect the model's behavior. Such a measure is called interpretability-faithfulness and is challenging to perform since the ground truth is inaccessible, and many LLMs only have an inference API. To address this, we propose employing …

abstract arxiv behavior confidence cs.ai cs.cl cs.lg excel instruction-tuned language language models large language large language models llms reasoning risk tasks type will