"transformers can use meaningless filler tokens (e.g., '......') in place of a chain of thought" - Let's Think Dot by Dot [P]

[https://arxiv.org/abs/2404.15758](https://arxiv.org/abs/2404.15758)

# From the abstract


We show that transformers can use meaningless filler tokens (e.g., '......') in place of a chain of thought to solve two hard algorithmic tasks they could not solve when responding without intermediate tokens. However, we find empirically that learning to use filler tokens is difficult and requires specific, dense supervision to converge

abstract chain of thought converge however intermediate machinelearning show solve supervision tasks thought tokens transformers