Think before you speak: Training Language Models With Pause Tokens

arXiv:2310.02226v2 Announce Type: replace-cross
Abstract: Language models generate responses by producing a series of tokens in immediate succession: the $(K+1)^{th}$ token is an outcome of manipulating $K$ hidden vectors per layer, one vector per preceding token. What if instead we were to let the model manipulate say, $K+10$ hidden vectors, before it outputs the $(K+1)^{th}$ token? We operationalize this idea by performing training and inference on language models with a (learnable) $\textit{pause}$ token, a sequence of which is appended to …

abstract arxiv cs.ai cs.cl cs.lg generate hidden language language models layer per responses series speak succession think token tokens training type vector vectors