Meet Quiet-STaR - AI Learnt to THINK before SPEAKING!!!

From abstract:

Quiet-STaR, a generalization of STaR in which LMs learn to
generate rationales at each token to explain future text, improving their
predictions. We address key challenges, including 1) the computational cost
of generating continuations, 2) the fact that the LM does not initially know
how to generate or use internal thoughts, and 3) the need to predict beyond
individual next tokens. To resolve these, we propose a tokenwise parallel
sampling algorithm, using learnable tokens indicating a thought’s start …

abstract challenges computational cost future generate key learn lms predictions speaking star text think token