Alphazero-like Tree-Search can Guide Large Language Model Decoding and Training

Recent works like Tree-of-Thought (ToT) and Reasoning via Planning (RAP) aim to augment the reasoning capabilities of LLMs by using tree-search algorithms to guide multi-step reasoning. These methods rely on prompting a pre-trained model to serve as a value function and focus on problems with low search depth. As a result, these methods will not work in domains where the pre-trained LLM does not have enough knowledge to serve as an effective value function or in domains that require long-horizon …

aim algorithms alphazero capabilities cs.ai cs.cl cs.lg decoding focus function guide language language model large language large language model llms low planning prompting rap reasoning search search algorithms serve thought training tree value via