CodeIt: Self-Improving Language Models with Prioritized Hindsight Replay

Large language models are increasingly solving tasks that are commonly believed to require human-level reasoning ability. However, these models still perform very poorly on benchmarks of general intelligence such as the Abstraction and Reasoning Corpus (ARC). In this paper, we approach ARC as a programming-by-examples problem, and introduce a novel and scalable method for language model self-improvement called Code Iteration (CodeIt). Our method iterates between 1) program sampling and hindsight relabeling, and 2) learning from prioritized experience replay. By relabeling …

abstraction arc benchmarks cs.ai cs.cl cs.lg examples general human intelligence language language models large language large language models novel paper programming reasoning scalable tasks