Prompting Large Language Models for Counterfactual Generation: An Empirical Study

arXiv:2305.14791v2 Announce Type: replace
Abstract: Large language models (LLMs) have made remarkable progress in a wide range of natural language understanding and generation tasks. However, their ability to generate counterfactuals has not been examined systematically. To bridge this gap, we present a comprehensive evaluation framework on various types of NLU tasks, which covers all key factors in determining LLMs' capability of generating counterfactuals. Based on this framework, we 1) investigate the strengths and weaknesses of LLMs as the counterfactual generator, …

abstract arxiv bridge counterfactual cs.cl evaluation framework gap generate language language models language understanding large language large language models llms natural natural language nlu progress prompting study tasks type types understanding