[D] Giving Autoregressive Models the Space to Think

Autoregressive prediction has a problem: whether you're asking what color the sky is or to prove the Riemann hypothesis, the amount of compute to generate the next token is the exact same, but it seems obvious which of the two questions requires more compute to answer. So, engineers toil on how to extend an autoregressive model's capabilities to be able to think, for a variable amount of time, before speaking. Here is the solution (exercise left to the reader).

 

A …

autoregressive models color compute engineers generate giving hypothesis machinelearning next prediction prove questions space think token