Markovian Agents for Truthful Language Modeling

arXiv:2404.18988v1 Announce Type: new
Abstract: Chain-of-Thought (CoT) reasoning could in principle enable a deeper understanding of a language model's (LM) internal reasoning. However, prior work suggests that some LMs answer questions similarly despite changes in their CoT, suggesting that those models are not truly using the CoT. We propose a training method to produce CoTs that are sufficient alone for predicting future text, independent of other context. This methodology gives a guarantee that if the LM can predict future tokens, …

abstract agents arxiv cs.cl however language language model lms modeling prior questions reasoning thought training type understanding work