L2MAC: Large Language Model Automatic Computer for Extensive Code Generation

arXiv:2310.02003v3 Announce Type: replace-cross
Abstract: Transformer-based large language models (LLMs) are constrained by the fixed context window of the underlying transformer architecture, hindering their ability to produce long and coherent outputs. Memory-augmented LLMs are a promising solution, but current approaches cannot handle long output generation tasks since they (1) only focus on reading memory and reduce its evolution to the concatenation of new memories or (2) use very specialized memories that cannot adapt to other domains. This paper presents L2MAC, …

abstract architecture arxiv augmented llms code code generation computer context context window cs.ai cs.lg cs.pl cs.se current language language model language models large language large language model large language models llms memory solution tasks transformer transformer architecture type