Reframing Human-AI Collaboration for Generating Free-Text Explanations. (arXiv:2112.08674v2 [cs.CL] UPDATED)

Web: http://arxiv.org/abs/2112.08674

Large language models are increasingly capable of generating fluent-appearing
text with relatively little task-specific supervision. But can these models
accurately explain classification decisions? We consider the task of generating
free-text explanations using human-written examples in a few-shot manner. We
find that (1) authoring higher quality prompts results in higher quality
generations; and (2) surprisingly, in a head-to-head comparison, crowdworkers
often prefer explanations generated by GPT-3 to crowdsourced explanations in
existing datasets. Our human studies also show, however, that while models …

ai arxiv collaboration free human text