LLMs as General Pattern Machines [R]

Full text: [https://arxiv.org/abs/2307.04721](https://arxiv.org/abs/2307.04721)

​

Abstract:

​

>We observe that pre-trained large language models (LLMs) are capable of autoregressively completing complex token sequences -- from arbitrary ones procedurally generated by probabilistic context-free grammars (PCFG), to more rich spatial patterns found in the Abstraction and Reasoning Corpus (ARC), a general AI benchmark, prompted in the style of ASCII art. Surprisingly, pattern completion proficiency can be partially retained even when the sequences are expressed using tokens randomly sampled from the vocabulary. These results …

abstract abstraction ai benchmark arc art benchmark context found free general general ai generated language language models large language large language models llms machinelearning observe patterns reasoning spatial token