Honest Students from Untrusted Teachers: Learning an Interpretable Question-Answering Pipeline from a Pretrained Language Model. (arXiv:2210.02498v1 [cs.CL])

Explainable question answering systems should produce not only accurate
answers but also rationales that justify their reasoning and allow humans to
check their work. But what sorts of rationales are useful and how can we train
systems to produce them? We propose a new style of rationale for open-book
question answering, called \emph{markup-and-mask}, which combines aspects of
extractive and free-text explanations. In the markup phase, the passage is
augmented with free-text markup that enables each sentence to stand on its …

arxiv language language model pipeline pretrained language model students