Tailoring Self-Rationalizers with Multi-Reward Distillation

arXiv:2311.02805v2 Announce Type: replace
Abstract: Large language models (LMs) are capable of generating free-text rationales to aid question answering. However, prior work 1) suggests that useful self-rationalization is emergent only at significant scales (e.g., 175B parameter GPT-3); and 2) focuses largely on downstream performance, ignoring the semantics of the rationales themselves, e.g., are they faithful, true, and helpful for humans? In this work, we enable small-scale LMs (approx. 200x smaller than GPT-3) to generate rationales that not only improve downstream …

abstract arxiv cs.cl distillation free gpt gpt-3 however language language models large language large language models lms performance prior question question answering replace semantics text type work