The Unreliability of Explanations in Few-Shot In-Context Learning. (arXiv:2205.03401v1 [cs.CL])

Web: http://arxiv.org/abs/2205.03401

How can prompting a large language model like GPT-3 with explanations improve
in-context learning? We focus specifically on two NLP tasks that involve
reasoning over text, namely question answering and natural language inference.
Including explanations in the prompt and having the model generate them does
not consistently improve performance in the settings we study, contrary to
recent results on symbolic reasoning tasks (Nye et al., 2021; Wei et al.,
2022). Despite careful prompting, explanations generated by GPT-3 may not even …

arxiv context learning