[R] Diffusion of Thoughts: Chain-of-Thought Reasoning in Diffusion Language Models

**Paper**: [https://arxiv.org/abs/2402.07754](https://arxiv.org/abs/2402.07754)

**Code**: [https://github.com/HKUNLP/diffusion-of-thoughts](https://github.com/HKUNLP/diffusion-of-thoughts)

**Abstract**:

>Diffusion models have gained attention in text processing, offering many potential advantages over traditional autoregressive models. This work explores the integration of diffusion models and Chain-of-Thought (CoT), a well-established technique to improve the reasoning ability in autoregressive language models. We propose **Diffusion-of-Thought** (**DoT**), allowing reasoning steps to diffuse over time through the diffusion process. In contrast to traditional autoregressive language models that make decisions in a left-to-right, token-by-token manner, DoT offers more flexibility in the …

abstract advantages attention autoregressive models contrast diffusion diffusion models integration language language models machinelearning process processing reasoning text thought through work